Guide · streaming

Stream a file at full speed

Send a file's bytes through a realtime session and read the transcript as fast as the server makes it. Nothing is stored.

A realtime session can take a whole file instead of a live microphone: you send the file's own bytes (mp3, wav, flac, ogg, m4a, a video…), the server decodes them and transcribes at full speed, many times faster than the audio plays, and the transcript comes back over the same socket. Nothing is stored. When you want the transcript kept, shareable and exportable, create a transcription instead (Transcribe a file).

1. You send: POST /v1/realtime/sessions (input: { kind: "file" })
2. VocalStack sends: { streamUrl }
3. You send: open a WebSocket to streamUrl
4. VocalStack sends: session.ready
5. You send: the file's bytes (binary frames, as fast as the socket takes them)
6. You send: {"type":"eof"} (transcription starts here)
7. VocalStack sends: transcript.progress (every few seconds)
8. VocalStack sends: transcript.committed (appends a final segment)
9. VocalStack sends: session.done

1. Create a file session

   POST /v1/realtime/sessions with input: { kind: "file" } and the usual options (spokenLanguage, translate, speakerLabels). A file session sends no drafts unless you ask (events: { drafts: true }): a draft previews a live tail, and a file has none. The file's decoded length must fit limits.maxDurationS (GET /v1/realtime/capabilities); a longer one is refused with invalid_audio.

2. Send the bytes

   Open the socket, wait for session.ready, then send the file as binary frames in chunks (the SDKs send 256 KB at a time and hold back while the socket's buffer is full), followed by {"type":"eof"}. Transcription starts at eof: no transcript frame arrives before it. vs.realtime.transcribeFile (TypeScript) and vs.realtime.transcribe_file (Python) do all of this in one call.

3. Follow the progress

   transcript.progress arrives on a timer (it is also the keep-alive) with processedS, the media seconds transcribed so far, and totalS once the file's length is known. Commits arrive as they are made, in order, with gapless seq; append them.

   {"type":"session.ready"}
   {"type":"transcript.progress","processedS":0,"totalS":184.2}
   {"type":"transcript.progress","processedS":31.5,"totalS":184.2}
   {"type":"transcript.committed","segment":{"seq":0,"start":0,"end":4.2,"text":"Welcome back to the show.","speaker":0,"words":[{"start":0,"end":0.4,"text":"Welcome"},{"start":0.4,"end":0.7,"text":"back"},{"start":0.7,"end":0.9,"text":"to"},{"start":0.9,"end":1,"text":"the"},{"start":1,"end":1.4,"text":"show."}]}}
   {"type":"transcript.committed","segment":{"seq":1,"start":4.6,"end":9.8,"text":"Today we are talking about rivers.","speaker":0,"words":[{"start":4.6,"end":4.9,"text":"Today"},{"start":4.9,"end":5.1,"text":"we"},{"start":5.1,"end":5.3,"text":"are"},{"start":5.3,"end":5.8,"text":"talking"},{"start":5.8,"end":6.1,"text":"about"},{"start":6.1,"end":6.7,"text":"rivers."}]}}
   {"type":"transcript.progress","processedS":96,"totalS":184.2}
   {"type":"transcript.progress","processedS":184.2,"totalS":184.2}
   {"type":"session.done","reason":"eof","stats":{"audioS":184.2,"processingS":6.1,"realtimeFactor":30.2}}

4. Finish

   After the last commit the server sends session.done: reason is eof, or max_duration when a file with no declared length ran past the limit and was cut, and stats.realtimeFactor says how much faster than real time it ran. The socket then closes.

import { VocalStack } from '@vocalstack/sdk';

const vs = new VocalStack({ baseUrl: 'https://api.vocalstack.com', apiKey: process.env.VOCALSTACK_API_KEY });

// transcribeFile creates the file session, sends the bytes and eof, and
// collects what comes back. Nothing is stored.
const result = await vs.realtime.transcribeFile('./talk.mp3', {
  spokenLanguage: 'en',
  speakerLabels: 'on',
  onProgress: ({ processedS, totalS }) => console.log(`${processedS.toFixed(0)} s of ${totalS?.toFixed(0) ?? '?'} s`),
  onCommit: (segment) => console.log(`#${segment.seq} [${segment.speaker}] ${segment.text}`),
});

console.log(result.text);
console.log(`${result.doneReason}: ${result.stats?.realtimeFactor}× real time`);

import asyncio
import os

from vocalstack import VocalStack


async def main() -> None:
    async with VocalStack("https://api.vocalstack.com", api_key=os.environ["VOCALSTACK_API_KEY"]) as vs:
        # transcribe_file creates the file session, sends the bytes and eof,
        # and collects what comes back. Nothing is stored.
        result = await vs.realtime.transcribe_file(
            "./talk.mp3",
            spoken_language="en",
            speaker_labels="on",
            on_progress=lambda p: print(f"{p.processed_s:.0f} s of {p.total_s or 0:.0f} s"),
            on_commit=lambda s: print(f"#{s.seq} [{s.speaker}] {s.text}"),
        )

        print(result.text)
        print(result.done_reason, result.stats)


asyncio.run(main())

// No SDK: fetch for the session, the built-in WebSocket for the bytes (Node 22 or newer).
import { createReadStream } from 'node:fs';

const API = 'https://api.vocalstack.com';

const res = await fetch(`${API}/v1/realtime/sessions`, {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.VOCALSTACK_API_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({ input: { kind: 'file' }, spokenLanguage: 'en' }),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
const { streamUrl } = (await res.json()) as { streamUrl: string };
const ws = new WebSocket(streamUrl);

ws.addEventListener('message', async (m: MessageEvent) => {
  if (typeof m.data !== 'string') return;
  const f = JSON.parse(m.data);
  if (f.type === 'session.ready') {
    for await (const chunk of createReadStream('./talk.mp3', { highWaterMark: 256 * 1024 })) {
      while (ws.bufferedAmount > 4 * 1024 * 1024) await new Promise((r) => setTimeout(r, 10)); // let the socket drain
      ws.send(new Uint8Array(chunk as Buffer));
    }
    ws.send(JSON.stringify({ type: 'eof' })); // transcription starts here
  }
  if (f.type === 'transcript.progress') console.log(`${f.processedS} s of ${f.totalS ?? '?'} s`);
  if (f.type === 'transcript.committed') console.log(`#${f.segment.seq} ${f.segment.text}`);
  if (f.type === 'session.done') console.log(f.reason, f.stats);
  if (f.type === 'error') console.error(f.code, f.message);
});