ink-preview transcribes 44 languages and detects the language automatically, even when a speaker mixes languages in one sentence. If you know which languages to expect, pass them as hints when you connect:
ink-2 supports English, French, Hindi, Japanese, and Spanish, and detects them automatically. Language hints and detected languages are available on ink-preview only.Supported languages
Use these codes for language hints. Detected languages use the same codes.Language hints
Language hints bias the model toward the languages you expect. The model can still transcribe other languages, and without hints it detects the language automatically. Hints can improve transcription accuracy, so we recommend adding them whenever you know the languages in advance.- Pass one
language_hintquery parameter per language, up to 6. In the URL,language_hintsalso works. In aconfigmessage, uselanguage_hints. Alanguage_hintfield there is ignored. - Use the base codes from the table above, such as
pt, not regional codes such aspt-BR. - If you connect with an unsupported code or more than 6 hints, you get an
errorevent and the connection closes. ink-2doesn’t take hints. It returns anerrorevent if you pass them.- With
ink-2andink-previewon Realtime STT (Manual), leave out thelanguagequery parameter. Onlyink-whisperuses it. If you pass it anyway, the value must be a languageink-whispersupports.
Change hints mid-stream
Withink-preview, send a config message to replace the hints without reconnecting. This works on both realtime endpoints:
- The new list replaces the old one. Send
[]to clear it, or leave the field out to keep the current list. - One message can change
keytermsandlanguage_hintstogether. On Realtime STT (Auto), it can also changeturnsettings, on any model. - The change takes effect right away, so it also applies to words not returned yet. Those can include speech from just before the message.
- The message is checked against the same limits as at connect time. There’s no reply when it works. If anything is wrong, you get an
errorevent, nothing changes, and the connection stays open.
Detected languages
On Realtime STT (Auto),turn.eager_end and turn.end include a languages list: the languages detected in the most recent audio.
The language list is sorted by confidence, so the first entry is the language the model is most sure of. Use it when you need one language, for example to choose which language your agent replies in.
The field can be missing, such as on
ink-2 or when a turn has too little audio to detect a language, so check for it first:
flush_done lists languages at each finalize, and done lists them when the session ends. Both look at recent audio, so they can include speech from before the previous finalize. If no audio arrived since the previous finalize, the event has no languages. Silence counts as audio too, and with no speech the list can name languages nobody spoke.
Good to know
-
turn.eager_endreports languages before the turn is over, andturn.endreports them again. The two can differ. If you start a reply onturn.eager_end, check the language again atturn.end, and regenerate the reply if it changed. -
Hints don’t limit detection to the hinted languages. A turn hinted
hican come back asenif the caller spoke English.
Where to go next
Word timestamps
Start and end times for every word
Keyterm prompting
Improve accuracy on names and rare words
API reference
Every event and field