Speech-to-text

5.1 Batch: POST /stt

multipart/form-data with field audio (≤ 25 MB, any common container: WAV at any rate, bit depth and channel count, MP3, Ogg/Opus, Ogg/Vorbis, FLAC, M4A/AAC, WebM, AIFF, A-law and µ-law WAV, or a video file with an audio track; the content decides, not the file name; headerless PCM has to be wrapped in a WAV header, or sent to the realtime socket), optional lang (one code, several candidates or auto, see 5.3; default de-CH), vocabulary (up to 200 domain terms of up to 100 characters to bias recognition towards: names, products, place names; a JSON array or a comma-separated list), swiss_german_detection (true to be told whether Swiss German or High German was spoken, on request, see 5.3) and conversation_id (one decision per call, see 5.3).

curl -X POST https://api.suisse-speech.ch/v1/stt \
  -H "X-API-Key: $SUISSE_SPEECH_KEY" \
  -F audio=@call.wav -F lang=de-CH \
  -F 'vocabulary=["Rüegg","Bircher","Glasfaser"]'

{"text":"…","lang":"de-CH","words":[{"w":"Grüezi","start":0.12,"end":0.58,"conf":0.98}],"duration_s":3.4}

duration_s is the length of the audio, silence included, and is what you are billed for. words[] joined with spaces is text split on whitespace, and the time stamps go forward and lie inside the audio.

POST /stt takes up to 4 minutes of audio and answers in about the length of the recording. Longer audio is refused at once with 413 stt.audio_too_large and a detail saying so. For those, submit a recognition job instead: it accepts up to 200 MB and runs at roughly a third of real time.

Everything is checked before recognition starts and refused with a detail rather than applied in part: a lang the catalogue does not know (400 stt.invalid_request; it is never sent to the recogniser as a guess), more than eight candidates, a 201st vocabulary term or a term over 100 characters, a recording shorter than 0.1 s (400 stt.invalid_audio), an upload over 25 MB (413, the limit is inclusive), a file that is not audio (415 stt.unsupported_format). GET /capabilities lists these limits under limits (stt_max_upload_bytes, stt_max_audio_seconds) and the recognition error codes under errors.

5.2 Realtime: GET /stt/stream (WebSocket)

Endpoint: wss://api.suisse-speech.ch/v1/stt/stream

{"type":"config","lang":"de-CH","vocabulary":["Bircher","Rüegg"]}
{"type":"audio.append","data":"<base64 PCM16 16 kHz mono>"}
{"type":"audio.end"}

Binary frames are accepted as raw PCM16 (they need not end on a sample boundary); audio that arrives before config or before session.ready is kept and recognised in order. The server replies with session.ready, then transcript.partial every 250 ms (the first one as soon as the recogniser has heard a word, which can be before the first tick), then transcript.final, then session.complete and a close with code 1000; after an error frame it closes with 1011. Every transcript.partial and transcript.final carries lang, the language of that stretch of speech. A lang the catalogue does not know (GET /capabilities lists them) is answered with an error frame, stt.invalid_request, before any recognition starts. The upgrade response carries X-Suisse-Environment like every other answer.

Send audio at real-time pace. Sending faster does not produce transcripts faster, and it will exhaust your budget for no benefit.

What a transcript.final contains

A final is a stretch of speech the recogniser has committed to. It always ends on a word boundary: a word is never split across two finals, and every entry in words is a whole word with the timings of that whole word. The one exception is yours to make: a finalize sent in the middle of a word (below) cuts the word there.

How much speech one final carries is up to the recogniser: sometimes a word, sometimes a whole sentence, often a few words from the middle of one. By default finals do not align with sentences and are not a sentence boundary you can build on (with finalize_on_pause, below, they mostly do). What always holds: finals arrive in order, no word appears in two of them, and concatenating their text in order gives the transcript.

Getting words sooner

Without a commit the recogniser decides on its own schedule, which keeps as much context as it can: a word is typically final about five to six seconds after it was spoken. Partials are there the whole time, every 250 ms. A voice bot commits at the end of every turn with finalize (below). For live captions and live translation of continuous speech, let the stream commit at pauses instead:

{"type":"config","lang":"de-CH","finalize_on_pause":true}

With finalize_on_pause a sentence is committed as soon as the speaker pauses after it and the recogniser holds it to be complete, and anything pending is committed after 0.8 s of pause. Nothing is committed in the middle of a word. Finals produced this way carry "forced": true:

{"type":"transcript.final","text":"Diese Geschichten sind einfach, aber sie haben es in sich.","forced":true,…}

The punctuation of a forced final is the recogniser’s reading of the speech. A commit does not add a sentence end of its own, so a trailing full stop means the sentence is over, and a forced final without one means the sentence goes on in the next final.

Measured on 2.5 minutes of a Swiss German sermon (also with background noise and room echo), on Standard German, French and six dialects: a sentence was final 0.7 s after its last word (median; 6 s by default). No word was split across two finals, the commits added no sentence ends, and word accuracy against the spoken text was on a par with the default.

Measured on a continuously speaking Bernese preacher (2.5 minutes): at 800 ms 11 of 60 finals were commits and a word was final 5.2 s after it was spoken (median), with every sentence end kept; at 600 ms 18 commits, 4.7 s and 12 of 15 sentence ends; at 400 ms 34 commits and 2.5 s, but only 3 of 15 sentence ends, because the commit comes before the recogniser has decided the sentence is over. If your pipeline reads punctuation, treat 600 as the floor; 400 suits captions where latency counts more than sentence ends.

At most one commit a second reaches the recogniser. Use partials, not finals, if you need something on screen within a quarter of a second.

Voice bots: finalize at once

A voice bot knows when the caller has stopped speaking: its own voice activity detection says so. Keep one stream open for the whole call, send the caller’s audio as it comes, and send {"type":"finalize"} at the end of every turn: the commit goes at once, and the finals of everything heard so far follow as soon as the recogniser confirms it. That is the default (finalize_mode "immediate"):

{"type":"config","lang":"de-CH","finalize_ack":true}

How soon. The last final of a turn arrives 0.03 s after your finalize (median; 0.07 s at the 95th percentile), whether your audio stopped or went on, as soon as after audio.end (measured on the service, without the network between you and us, on 2,400 telephone turns in October 2026; "pause" took 1.4 s when the audio stopped and 0.6 s when it went on). Counted from the caller’s last word, add the silence your voice activity detection waits for before it sends finalize. If you would rather not detect the end of a turn yourself, endpointing (above) lets the recogniser commit at the end of each turn; the final then comes once the recogniser holds the turn to be over, at most endpoint_delay_ms after the speech.

Send finalize where speech has ended, after your voice activity detection has heard about 200 ms of silence. With no silence at all the last word was missed or came late about twice as often as with 100 ms or more. A finalize in the middle of a word cuts the word there: its first part ends one final and the rest begins the next. The finals of a commit carry "forced": true. Sent after at least 200 ms of silence, the commit keeps the sentence end the recogniser gives it, so the last final of a turn ends as it would with audio.end (in our measurements in 95 % of turns after 300 ms of silence; after 200 ms in 96 % when the audio stopped and 74 % when it went on). With less silence, a sentence end the recogniser only added because of the commit is removed, and the final usually ends without a full stop.

"finalize_ack": true answers every finalize with exactly one event, after the finals of its commit, so your bot knows the turn is complete:

{"type":"finalize.done","request_id":"req_…","committed":true,"ms":38}

committed: false means there was nothing left to commit (no audio since the previous commit), or audio.end took it, in which case the event comes before session.complete. ms is the time from your finalize to this event on our side. Without finalize_ack no such event is sent. session.ready names the mode in force, and says "finalize_ack": true when the events will come:

{"type":"session.ready","request_id":"req_…","finalize_mode":"immediate","finalize_ack":true}

Any other value of finalize_mode is ignored: the default applies, as without the field. With Swiss German detection a German final can still wait for its variety (below; with conversation_id only until the call is decided), and finalize.done comes after it: nothing overtakes a final.

One stream for the call, one finalize per turn:

import WebSocket from 'ws';

const ws = new WebSocket('wss://api.suisse-speech.ch/v1/stt/stream', {
  headers: { 'X-API-Key': process.env.SUISSE_SPEECH_KEY },
});
ws.on('open', () => ws.send(JSON.stringify({ type: 'config', lang: 'de-CH,fr-CH,en', finalize_ack: true })));

const send = (frame) => { if (ws.readyState === WebSocket.OPEN) ws.send(frame); };
call.onAudio((pcm16k) => send(pcm16k));                                    // the caller's audio, the whole call
vad.on('end-of-speech', () => send(JSON.stringify({ type: 'finalize' })));  // about 200 ms of silence

let turn = [];
ws.on('message', (data, isBinary) => {
  if (isBinary) return;
  const msg = JSON.parse(data.toString());
  if (msg.type === 'transcript.final') turn.push(msg);
  if (msg.type === 'finalize.done') {          // every final of the turn is in
    if (turn.length) reply(turn.map((f) => f.text).join(' '), turn.at(-1).lang);
    turn = [];
  }
});

Knowing the session is working

A stretch without transcript.partial does not by itself mean a fault: music, a hymn, a pause or a signal too faint to hold speech give no words either. To tell these apart from a session that has stopped, ask for a status in config:

{"type":"config","lang":"de-CH","status_interval_ms":5000}

status_interval_ms (1000–60000, off by default) sends a session.status at that interval from session.ready until the end of the session:

{"type":"session.status","request_id":"req_…","audio_ms":184000,"processed_ms":183920,"last_word_ms_ago":1250,"stalled":false}

processed_ms keeping up while last_word_ms_ago grows: the recogniser hears no speech in what you send, so look at the audio source. stalled: true: reconnect and continue with a new session. No session.status at all: the connection itself is gone. A value out of range is refused with stt.invalid_request.

5.3 Multilingual recognition

lang takes one code, several candidates separated by commas, or auto. It works the same way on POST /stt, on /stt/stream and on recognition jobs.

Any of the 60 languages of the catalogue can be a candidate, with the same codes synthesis accepts (Text-to-speech 3.4).

With more than one candidate the spoken language is detected automatically, the transcript is written in that language, and every result reports it in lang. On the stream each transcript.partial and transcript.final carries the lang of that stretch of speech, so a change of language during a session is visible as it happens.

The reported value is always one of the candidates you gave. Two things follow:

• You never have to handle a language you did not list. Name only the languages you expect: fewer candidates give a steadier result on very short utterances.
• A candidate may carry a region or a dialect (fr-CH, de-CH-zurich) and is reported back exactly so.

Swiss German dialects and High German are both recognised under de-CH (or de) and both are written as Standard German, which is what downstream language understanding handles best. Swiss German can also be named with its BCP 47 code: gsw, gsw-CH and gsw-CH-zurich mean exactly what de-CH and de-CH-zurich mean, are reported back as given, and are accepted by synthesis.

Recognition reports the language, not the regional dialect that was spoken. Whether Swiss German or High German was spoken is reported when you ask for it (below); otherwise two candidates that share a base language (de-CH,de) cannot be told apart, and the one listed first is reported. Which variety to answer in is your choice (5.4).

Swiss German or High German (on request)

Recognition can also tell whether Swiss German (dialect) or High German was spoken, so that you can answer in the same variety. It is available on request: write to us at info@suisse-it.ch or through the contact form, and we enable it for your account. It is then off unless you send "swiss_german_detection": true (in config, or a form field on POST /stt and POST /jobs).

{"type":"config","lang":"de-CH,de-DE","swiss_german_detection":true}

Every transcript.final in German then carries variety, and with a Swiss German and a High German candidate listed, lang reports the candidate of the variety heard:

{"type":"transcript.final","text":"Ich möchte mein Abo kündigen.","lang":"de-CH",
 "variety":{"spoken":"swiss_german","confidence":0.9,"decided":true},"words":[…],"duration_s":3.1}

• spoken: swiss_german or high_german. High German spoken with a Swiss accent counts as high_german, but is reported as swiss_german about half the time (accuracy below).
• confidence: how sure the decision is, from 0.5 to 1.
• decided: false when there was too little speech to tell (a single word such as “ja” or “merci”: under about one second of its own speech a final takes the decision of the speech just before it, if there is one) or the audio was unclear. lang is then the first German candidate you listed, exactly as without detection, and spoken is only a lean. List the variety you are currently answering in first, and a short answer keeps it.
• source (only when you send conversation_id, see below): "utterance" when this result was judged on its own, "call" when it carries the decision the call already had.

Accuracy, measured per utterance on telephone calls of 202 real callers: Swiss German (dialect) was reported as swiss_german in 92 % of the utterances, High German of speakers from Germany and Austria as high_german in 88 %. High German spoken by Swiss speakers was reported as swiss_german about half the time.

Only utterances recognised as German are checked. With a candidate list that also holds other languages ("de-CH,de-DE,en,fr-CH,it-CH"), an English, French or Italian utterance is never checked and carries no variety; its final waits only when it follows a German final whose decision is still pending, so that results stay in order. On POST /stt and recognition jobs a recording in another language is not checked, and one recognised as German is decided from its first seconds of speech. Partials carry no variety; the regional dialect is not detected.

Detection takes time for German. For a German utterance with less than about 4 s of speech the final can come up to about one second later (never more than max_final_delay_ms in GET /capabilities, today 1.5 s); longer utterances are usually not delayed, because the decision is made while the caller is still speaking. With conversation_id (below) the wait ends with the first decision of the call: every later German final of the call carries the decided variety at once. Leave detection off for streams where every millisecond counts and the variety does not matter.

Without "swiss_german_detection": true nothing changes: the first listed candidate is reported, as before, and nothing waits. With a single German code, variety is added and lang stays as given. Detection is enabled per account, on request (above); GET /capabilities says whether yours has it (speech_to_text.swiss_german_detection.enabled_for_your_account), and without it the field is refused with 400 stt.invalid_request and a detail (on the stream: an error frame before session.ready).

One decision per call

A caller practically never changes between Swiss German and High German within a call, so the variety is best decided once per call, not once per utterance. Send the same conversation_id with every request of a call: in the config of every stream, or as a form field on POST /stt.

{"type":"config","lang":"de-DE,de-CH","swiss_german_detection":true,"conversation_id":"call-7f3c2a1e"}

• Until a German result of the call is decided, each is judged on its own, as above, with "source":"utterance".
• The first decided result decides the call. Every later German result of the same call, in the same stream or in any later stream or recording, carries that variety at once: "decided":true, "source":"call", and lang the first listed candidate of that variety. These results are not delayed by detection, the voice of your answers no longer changes from utterance to utterance, and a short “ja” carries the caller’s variety too.
• While the decision is new, the caller’s next German utterances are still checked in the background, without delaying anything: once one agrees, the decision is settled and nothing more is checked; if the next two both disagree, it is corrected once, early in the call.
• conversation_id is your own identifier of the call (a UUID, your call or session id): 1 to 128 visible ASCII characters without spaces, or a whole number. Any other value is ignored, never refused: the request is then decided per utterance, as without the field, and its results carry no source. It is matched within your account only. We do not keep the identifier itself, only a one-way hash of it with the decided variety, for two hours after the decision.

For a voice bot this is the recommended setup: send conversation_id in the config of the call’s stream (and of every stream of the call, if you open more than one), and until the call is decided list first the variety you would rather answer in when unsure. If answering a High German speaker in dialect is worse for you than the reverse, list de-DE before de-CH ("de-DE,de-CH"): results that are not decided report the first German candidate listed.

Without conversation_id nothing changes: every utterance is judged on its own.

5.4 From recognition to synthesis: answering in the caller’s language

Both directions use the same codes, so the language a caller speaks is handed from one to the other unchanged:

1. Open recognition with your target locales as candidates: {"type":"config","lang":"de-CH-zurich,fr-CH,en"}
2. Read lang from transcript.final, for example "lang":"fr-CH".
3. Write your reply in that language and synthesise it with language set to the same value: {"type":"session.start","language":"fr-CH","voice":"anna", ...}

No mapping table and no server-side session is involved. Every request stands on its own (the one thing remembered is the variety decided for a call when you send conversation_id, 5.3), and the code carries everything synthesis needs, the dialect included. All voices speak every supported language, so one voice can serve the whole conversation and the persona stays the same when the language changes.

Choosing the variety you answer in:

• Swiss German: de-CH with a dialect, or the compact form de-CH-zurich. The reply may be written in Standard German; it is spoken in the dialect.
• High German: de-CH-hochdeutsch for Swiss High German, or de, de-DE, de-AT.
• Anything the catalogue does not list: describe it in style (Text-to-speech 3.6).

If the language is known before the conversation starts (a language menu, the customer record) pass that single code instead of a candidate list.