# Live translation and spoken translations > Translate a stream as it is spoken, and have each translated sentence spoken back in a voice. Text frames and audio clips, over the same socket. | Endpoint | What it does | |---|---| | `GET /v1/languages` | List languages | | `GET /v1/voices/available` | List available voices | | `GET /v1/recordings/{id}/voice/{sentence}` | Get a spoken sentence | | `GET /v1/recordings/{id}/voice` | Get a recording's voice tracks | | `WS /v1/realtime/sessions/{token}/socket` | Connect to the `streamUrl` a realtime session returned (treat it as opaque). | A realtime session (and every recording, for its listeners) can translate what is said into other languages as it goes, and speak the translation: each finished sentence is synthesised in the language's voice and sent as a short audio clip. Translations follow the transcript by a second or two; a spoken sentence follows its translation. 1. You → VocalStack: `options: translate ["es"], sentences { enabled: true }, voice ["es"]` 2. VocalStack → you: `transcript.committed` (the words as spoken) 3. VocalStack → you: `transcript.sentence (open)` (the sentence so far, a provisional translation) 4. VocalStack → you: `transcript.sentence (closed)` (the sentence and its final translation) 5. VocalStack → you: `transcript.voice` (that sentence, spoken in Spanish (Ogg/Opus)) ## 1. Pick the languages `GET /v1/languages` lists what can be transcribed (`transcription`), translated into (`translation`) and spoken (`voice`), with names. `GET /v1/voices/available?lang=es` lists the voices that speak a language, best first; `default` marks the one used when none is named, and with a key your own cloned voices are there too. ## 2. Ask for translations `translate: ["es", "fr"]` on the session's options. Without sentences, each commit is followed by a `transcript.translated` frame naming its `seq`: attach `translations` to that segment. You can change the list mid-stream (`session.update`, `translate: […]`); it applies to what follows. ## 3. Read them by sentence With `sentences: { enabled: true }`, the server also assembles commits into sentences and translates whole sentences, which reads far better. `transcript.sentence` is a full replacement of one sentence, keyed by `sentenceId`: `status` goes `open` → `closed` (or `dropped`: discard it), `seqs` lists the commits it covers, and `translations` are provisional while it is open and final once it closes. Upsert by `sentenceId`; never append. ## 4. Speak them `voice: ["es"]` (it needs `sentences`) speaks each closed sentence's Spanish translation in the language's default voice; `"es:"` names a voice from the catalogue, and `voiceByLang: { "es": "" }` sets one per language. With `voiceCast` on (the default), each speaker gets a distinct voice of matching gender. Each clip arrives as a `transcript.voice` frame: `sentenceId`, `lang`, `durationS`, and `audio`, the clip itself (base64 Ogg/Opus, 24 kHz mono). A long sentence comes in pieces (`part` of `parts`) so the first can play while the rest are made: play them back to back, in order. ```text +0.0s {"type":"session.ready"} +2.6s {"type":"transcript.committed","segment":{"seq":0,"start":0.0,"end":1.4,"text":"Good morning, everyone.","speaker":0,"words":[]}} +2.7s {"type":"transcript.sentence","sentence":{"sentenceId":0,"status":"open","text":"Good morning, everyone.","seqs":[0],"speaker":0,"start":0.0,"end":1.4,"translations":{"es":"Buenos días a todos."}}} +3.9s {"type":"transcript.sentence","sentence":{"sentenceId":0,"status":"closed","text":"Good morning, everyone.","seqs":[0],"speaker":0,"start":0.0,"end":1.4,"translations":{"es":"Buenos días a todos."}}} +5.0s {"type":"transcript.voice","sentenceId":0,"generation":0,"lang":"es","voice":"es_ES-davefx-medium","format":"ogg","durationS":1.3,"synthS":0.4,"part":0,"parts":1,"voiceKey":"","audio":"T2dnUwACAAAA…"} ``` ## 5. For a recording's listeners A listener on the subscribe socket asks with `lang=es` (translations filtered to Spanish) and `voice=` ([Listen](https://api.vocalstack.com/docs/guides/listen.md)); a clip reaches every listener on that language and voice, synthesised once. On a recording, `transcript.voice` carries a `url` instead of `audio`: fetch it with `GET /v1/recordings/{id}/voice/{sentence}?lang=es` (Ogg/Opus; `format=caf` for Apple). `transcript.voiceStatus` says when a voice is still `preparing` (its clips are held and will come) or `unavailable`. After the recording, `GET /v1/recordings/{id}/voice` lists every clip left behind, language by language. ## Complete code Each program runs as it is once `VOCALSTACK_API_KEY` holds one of your API keys. ### TypeScript SDK: `translate.ts` ```typescript import { createReadStream } from 'node:fs'; import { writeFile } from 'node:fs/promises'; import { VocalStack, applySentence, type SentenceSegment } from '@vocalstack/sdk'; const vs = new VocalStack({ baseUrl: 'https://api.vocalstack.com', apiKey: process.env.VOCALSTACK_API_KEY }); const { voices } = await vs.voices.available('es'); const voice = voices.find((v) => v.default) ?? voices[0]; // A file at full speed here; a microphone works the same (Stream a microphone). const session = await vs.realtime.connect( { input: { kind: 'file' }, spokenLanguage: 'en', translate: ['es'], sentences: { enabled: true }, voice: [`es:${voice.id}`], }, { // The clips arrive as transcript.voice frames: onFrame sees every frame. onFrame: (f) => { if (f.type === 'transcript.voice' && f.audio) { void writeFile(`es-${f.sentenceId}-${f.part ?? 0}.ogg`, Buffer.from(f.audio, 'base64')); } }, }, ); void session.sendFile(createReadStream('./talk.mp3')); let sentences: SentenceSegment[] = []; for await (const ev of session) { if (ev.type !== 'transcript.sentence') continue; sentences = applySentence(sentences, ev.sentence); // upsert by sentenceId if (ev.sentence.status === 'closed') console.log(`${ev.sentence.text}\n es: ${ev.sentence.translations?.es}`); } console.log(`${sentences.length} sentences; the Spanish clips are es--.ogg`); ``` ### Python SDK: `translate.py` ```python import asyncio import base64 import os from vocalstack import VocalStack async def main() -> None: async with VocalStack("https://api.vocalstack.com", api_key=os.environ["VOCALSTACK_API_KEY"]) as vs: catalogue = await vs.voices.available("es") voice = next((v for v in catalogue.voices if v.default), catalogue.voices[0]) # A file at full speed here; a microphone works the same (Stream a microphone). session = await vs.realtime.connect( input={"kind": "file"}, spoken_language="en", translate=["es"], sentences={"enabled": True}, voice=[f"es:{voice.id}"], ) async with session: sender = asyncio.create_task(session.send_file("./talk.mp3")) async for event in session: if event.type == "transcript.sentence" and event.sentence.status == "closed": print(event.sentence.text) print(" es:", (event.sentence.translations or {}).get("es")) elif event.type == "transcript.voice" and event.audio: with open(f"es-{event.sentence_id}-{event.part or 0}.ogg", "wb") as f: f.write(base64.b64decode(event.audio)) await sender asyncio.run(main()) ``` ### HTTP + WebSocket: `translate-raw.ts` ```typescript // No SDK: fetch for the session, the built-in WebSocket for the stream (Node 22 or newer). import { createReadStream } from 'node:fs'; import { writeFile } from 'node:fs/promises'; const API = 'https://api.vocalstack.com'; const auth = { Authorization: `Bearer ${process.env.VOCALSTACK_API_KEY}` }; const cat = (await (await fetch(`${API}/v1/voices/available?lang=es`, { headers: auth })).json()) as { voices: { id: string; default: boolean }[] }; const voice = cat.voices.find((v) => v.default) ?? cat.voices[0]; const res = await fetch(`${API}/v1/realtime/sessions`, { method: 'POST', headers: { ...auth, 'Content-Type': 'application/json' }, body: JSON.stringify({ input: { kind: 'file' }, spokenLanguage: 'en', translate: ['es'], sentences: { enabled: true }, voice: [`es:${voice.id}`], }), }); if (!res.ok) throw new Error(`${res.status} ${await res.text()}`); const ws = new WebSocket(((await res.json()) as { streamUrl: string }).streamUrl); ws.addEventListener('message', async (m: MessageEvent) => { if (typeof m.data !== 'string') return; const f = JSON.parse(m.data); if (f.type === 'session.ready') { for await (const chunk of createReadStream('./talk.mp3', { highWaterMark: 256 * 1024 })) ws.send(new Uint8Array(chunk as Buffer)); ws.send(JSON.stringify({ type: 'eof' })); } if (f.type === 'transcript.sentence' && f.sentence.status === 'closed') console.log(`${f.sentence.text}\n es: ${f.sentence.translations?.es}`); if (f.type === 'transcript.voice' && f.audio) await writeFile(`es-${f.sentenceId}-${f.part ?? 0}.ogg`, Buffer.from(f.audio, 'base64')); if (f.type === 'session.done') ws.close(); }); ``` --- All guides: https://api.vocalstack.com/llms.txt. Every operation: https://api.vocalstack.com/llms-full.txt. The OpenAPI document, with every WebSocket frame under `x-websocket`: https://api.vocalstack.com/v1/openapi.json.