# Stream a microphone

> Live audio in, transcript events out, over one WebSocket. Drafts replace the live tail; commits append.

| Endpoint | What it does |
|---|---|
| `POST /v1/realtime/sessions` | Create a realtime session |
| `WS /v1/realtime/sessions/{token}/socket` | Connect to the `streamUrl` a realtime session returned (treat it as opaque). |

A realtime session is a signed `streamUrl` and one WebSocket. You send raw audio as it is captured; the server sends the transcript back as it is spoken, a few hundred milliseconds behind. Nothing is stored: the transcript is what the socket sends. To keep it, publish to a live transcription instead ([Go live](https://api.vocalstack.com/docs/guides/go-live.md)).

1. You → VocalStack: `POST /v1/realtime/sessions` (with your API key, on your server)
2. VocalStack → you: `{ sessionId, streamUrl }`
3. You → VocalStack: `open a WebSocket to streamUrl`
4. VocalStack → you: `session.loading … session.ready` (a cold machine warms up first)
5. You → VocalStack: `binary PCM, every ~100 ms`
6. VocalStack → you: `transcript.draft` (replaces the live tail)
7. VocalStack → you: `transcript.committed` (appends a final segment)
8. You → VocalStack: `{"type":"eof"}`
9. VocalStack → you: `session.done` (then the socket closes)

## 1. Create a realtime session

`POST /v1/realtime/sessions` takes the session's options and answers `{ sessionId, streamUrl, expiresAt }`. `streamUrl` carries its own signed token; connect before `expiresAt`. Options worth knowing: `spokenLanguage` (`en`, or absent to detect it), `translate` (target languages, such as `["es"]`), `speakerLabels` (`auto`, `on`, `off`) and `latencyMode` (`auto`, `fast`, `accuracy`).

This call takes an API key, so it belongs on your server. A web page asks your server for a session and connects to the `streamUrl` it gets back: the key never reaches the browser (the TypeScript tab's `browser.ts`).

## 2. Open the socket

Connect to `streamUrl` as it is, with no header: the token is in the URL. While a machine warms up the server sends `session.loading` heartbeats (`stage`: `download`, then `model`), then `session.ready`. Send audio only after `session.ready`.

The SDKs' `connect` resolves at `session.ready` and fails if the server goes quiet for 30 s (`readyDeadlineMs`): with no frame at all that is `capacity_exceeded` (every machine is busy), after loading frames `engine_unavailable`. A refused upgrade carries the error envelope: `401 invalid_token`, `410 session_expired`, `503 unavailable` (retry after `Retry-After`).

## 3. Send audio

Audio goes in as binary frames of raw PCM: 32-bit float little-endian, 16 kHz, mono (`pcm_f32le`). Send it as it is captured, in chunks of about 100 to 250 ms (6,400 to 16,000 bytes); there is no need to wait for the transcript between chunks. In a browser, `streamMicrophone` (`@vocalstack/sdk/mic`) asks for the microphone, resamples to 16 kHz, sends the chunks and the `clock` frames that let the server mark gaps when a tab is throttled.

You can change `spokenLanguage`, `translate`, `speakerLabels` or `latencyMode` mid-stream with a `session.update` text frame (`session.update(…)` in the SDKs); it applies to what follows.

## 4. Read the transcript

Text frames come back as JSON with a `type`. Two of them build the transcript, and mixing them up is the most common mistake:

- `transcript.draft` is the **live tail**: the words heard since the last commit, still changing. Each draft is a full replacement. Redraw the tail with it; never append it.
- `transcript.committed` is **final**: one segment, with word timings, numbered by `seq` (0, 1, 2… gapless). Append it, and drop the draft it covers: the next draft starts where the commit ended.

Around them: `transcript.translated` attaches translations to a committed `seq`; `transcript.amended` corrects a committed segment's text or speaker in place (same `seq`); `transcript.words` streams word timings for the tail. Ignore frame types you do not know: new ones are added without notice.

```text
+0.0s {"type":"session.loading","stage":"model"}
+1.2s {"type":"session.ready"}
+2.0s {"type":"transcript.draft","segments":[{"start":0.0,"text":"Good","speaker":0,"tentative":true}]}
+2.5s {"type":"transcript.draft","segments":[{"start":0.0,"text":"Good morning every","speaker":0,"tentative":true}]}
+3.1s {"type":"transcript.committed","segment":{"seq":0,"start":0.0,"end":1.4,"text":"Good morning, everyone.","speaker":0,"words":[{"start":0.0,"end":0.3,"text":"Good"},{"start":0.3,"end":0.8,"text":"morning,"},{"start":0.8,"end":1.4,"text":"everyone."}]}}
+3.4s {"type":"transcript.translated","seq":0,"translations":{"es":"Buenos días a todos."}}
+3.9s {"type":"transcript.draft","segments":[{"start":1.6,"text":"Thanks for","speaker":0,"tentative":true}]}
+4.6s {"type":"transcript.draft","segments":[{"start":1.6,"text":"Thanks for coming in","speaker":0,"tentative":true}]}
+5.3s {"type":"transcript.committed","segment":{"seq":1,"start":1.6,"end":3.0,"text":"Thanks for coming in today.","speaker":0,"words":[{"start":1.6,"end":1.9,"text":"Thanks"},{"start":1.9,"end":2.1,"text":"for"},{"start":2.1,"end":2.5,"text":"coming"},{"start":2.5,"end":2.6,"text":"in"},{"start":2.6,"end":3.0,"text":"today."}]}}
+5.5s {"type":"transcript.translated","seq":1,"translations":{"es":"Gracias por venir hoy."}}
+5.9s {"type":"session.done","reason":"eof","stats":{"audioS":3.2,"processingS":1.4,"realtimeFactor":2.3}}
```

## 5. Stop

Send `{"type":"eof"}` when the audio ends (`session.eof()` in the SDKs). The server transcribes what it still holds, sends the last commits, then `session.done` with `reason` (`eof`, `max_duration`, `cancelled`) and `stats` (`audioS`, `processingS`, `realtimeFactor`), and closes. `cancel` stops at once and drops the rest.

There is no resume: a dropped connection ends the session. To go on, create a new one. A session lasts at most `limits.maxDurationS` (`GET /v1/realtime/capabilities`), and its audio spends the account's credit at the plan's live rate (`GET /v1/billing`).

## Complete code

Each program runs as it is once `VOCALSTACK_API_KEY` holds one of your API keys.

### TypeScript SDK: `mic.ts`

```typescript
// Node: ffmpeg captures the microphone, the SDK streams it.
import { spawn } from 'node:child_process';
import { RealtimeSession, VocalStack } from '@vocalstack/sdk';

const vs = new VocalStack({ baseUrl: 'https://api.vocalstack.com', apiKey: process.env.VOCALSTACK_API_KEY });

const { sessionId, streamUrl } = await vs.realtime.createSession({ spokenLanguage: 'en', translate: ['es'] });

const session = await RealtimeSession.connect({
  sessionId,
  streamUrl,
  onLoading: (stage) => console.error(`warming up (${stage})…`),
});

// Raw PCM: f32le, 16 kHz, mono. macOS: -f avfoundation -i :0, Linux: -f pulse -i default.
const mic = spawn('ffmpeg', ['-loglevel', 'error', '-f', 'avfoundation', '-i', ':0', '-f', 'f32le', '-ac', '1', '-ar', '16000', '-']);
mic.stdout.on('data', (chunk: Buffer) => session.sendAudio(new Uint8Array(chunk)));

mic.on('close', () => session.eof()); // the server flushes, then sends session.done
process.once('SIGINT', () => mic.kill()); // Ctrl-C ends the audio

for await (const ev of session) {
  if (ev.type === 'transcript.draft') {
    // The live tail: replace it, never append it.
    process.stdout.write(`\r\x1b[K\x1b[2m${ev.segments.map((s) => s.text).join(' ')}\x1b[0m`);
  } else if (ev.type === 'transcript.committed') {
    // Final: append it (seq 0, 1, 2…), and the tail it covers is gone.
    process.stdout.write(`\r\x1b[K${ev.segment.text}\n`);
  } else if (ev.type === 'transcript.translated') {
    console.log(`  es: ${ev.translations.es}`);
  }
}

console.log(`done (${session.doneReason})`, session.stats);
```

### TypeScript SDK: `browser.ts`

```typescript
// In a web page: your server creates the session (the API key stays
// there), and the page streams the microphone to its streamUrl.
import { RealtimeSession, VocalStack } from '@vocalstack/sdk';
import { streamMicrophone } from '@vocalstack/sdk/mic';

// On your server, behind a route of yours (POST /api/transcribe, say):
const vs = new VocalStack({ baseUrl: 'https://api.vocalstack.com', apiKey: process.env.VOCALSTACK_API_KEY });
export async function createStream(): Promise<{ sessionId: string; streamUrl: string }> {
  const { sessionId, streamUrl } = await vs.realtime.createSession({ spokenLanguage: 'en' });
  return { sessionId, streamUrl };
}

// In the page, with what your route returned:
export async function transcribeMicrophone(
  stream: { sessionId: string; streamUrl: string },
  render: (committed: string, tail: string) => void,
): Promise<void> {
  const session = await RealtimeSession.connect(stream);
  const mic = await streamMicrophone(session); // asks for the microphone, sends 16 kHz PCM
  document.querySelector('#stop')?.addEventListener('click', () => {
    void mic.stop();
    session.eof();
  });
  let committed = '';
  for await (const ev of session) {
    if (ev.type === 'transcript.draft') render(committed, ev.segments.map((s) => s.text).join(' '));
    if (ev.type === 'transcript.committed') {
      committed += `${ev.segment.text} `;
      render(committed, '');
    }
  }
}
```

### Python SDK: `mic.py`

```python
import asyncio
import os
import signal

from vocalstack import RealtimeSession, VocalStack

# Raw PCM: f32le, 16 kHz, mono. macOS: -f avfoundation -i :0, Linux: -f pulse -i default.
MIC = ["ffmpeg", "-loglevel", "error", "-f", "avfoundation", "-i", ":0",
       "-f", "f32le", "-ac", "1", "-ar", "16000", "-"]


async def main() -> None:
    async with VocalStack("https://api.vocalstack.com", api_key=os.environ["VOCALSTACK_API_KEY"]) as vs:
        created = await vs.realtime.create_session(spoken_language="en", translate=["es"])

        session = await RealtimeSession.connect(created.session_id, created.stream_url)

        mic = await asyncio.create_subprocess_exec(*MIC, stdout=asyncio.subprocess.PIPE)

        async def feed() -> None:
            assert mic.stdout is not None
            while chunk := await mic.stdout.read(8000):  # 125 ms of audio
                await session.send_audio(chunk)
            await session.eof()  # the server flushes, then sends session.done

        feeder = asyncio.create_task(feed())
        asyncio.get_running_loop().add_signal_handler(signal.SIGINT, mic.terminate)  # Ctrl-C ends the audio

        async with session:
            async for event in session:
                if event.type == "transcript.draft":
                    # The live tail: replace it, never append it.
                    tail = " ".join(s.text for s in event.segments)
                    print(f"\r\x1b[K\x1b[2m{tail}\x1b[0m", end="", flush=True)
                elif event.type == "transcript.committed":
                    # Final: append it (seq 0, 1, 2...), and the tail it covers is gone.
                    print(f"\r\x1b[K{event.segment.text}", flush=True)
                elif event.type == "transcript.translated":
                    print(f"  es: {event.translations.get('es')}")
            await feeder
            print(f"done ({session.done_reason})", session.stats)


asyncio.run(main())
```

### HTTP + WebSocket: `mic-raw.ts`

```typescript
// No SDK: fetch for the session, the built-in WebSocket for the stream
// (Node 22 or newer, or a browser once your server has created the session).
import { spawn } from 'node:child_process';

const API = 'https://api.vocalstack.com';

const res = await fetch(`${API}/v1/realtime/sessions`, {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.VOCALSTACK_API_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({ spokenLanguage: 'en', translate: ['es'] }),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`); // the error envelope: { code, message, … }
const { streamUrl } = (await res.json()) as { sessionId: string; streamUrl: string; expiresAt: string };

const ws = new WebSocket(streamUrl); // the token is in the URL: no header
ws.binaryType = 'arraybuffer';

ws.addEventListener('message', (m: MessageEvent) => {
  if (typeof m.data !== 'string') return;
  const f = JSON.parse(m.data);
  switch (f.type) {
    case 'session.loading':
      console.error(`warming up (${f.stage})…`);
      break;
    case 'session.ready':
      startMicrophone(); // send audio only after session.ready
      break;
    case 'transcript.draft': // the live tail: replace it
      process.stdout.write(`\r\x1b[K\x1b[2m${f.segments.map((s: { text: string }) => s.text).join(' ')}\x1b[0m`);
      break;
    case 'transcript.committed': // final: append it
      process.stdout.write(`\r\x1b[K${f.segment.text}\n`);
      break;
    case 'transcript.translated':
      console.log(`  es: ${f.translations.es}`);
      break;
    case 'session.done':
      console.log(`done (${f.reason})`, f.stats);
      break;
    case 'error':
      console.error(f.code, f.message);
      break;
  }
});

function startMicrophone(): void {
  // Raw PCM: f32le, 16 kHz, mono, as binary frames.
  const mic = spawn('ffmpeg', ['-loglevel', 'error', '-f', 'avfoundation', '-i', ':0', '-f', 'f32le', '-ac', '1', '-ar', '16000', '-']);
  mic.stdout.on('data', (chunk: Buffer) => ws.send(new Uint8Array(chunk)));
  mic.on('close', () => ws.send(JSON.stringify({ type: 'eof' })));
  process.once('SIGINT', () => mic.kill());
}
```


---

All guides: https://api.vocalstack.com/llms.txt. Every operation: https://api.vocalstack.com/llms-full.txt. The OpenAPI document, with every WebSocket frame under `x-websocket`: https://api.vocalstack.com/v1/openapi.json.