Skip to main content
ink-preview transcribes 44 languages and detects the language automatically, even when a speaker mixes languages in one sentence. If you know which languages to expect, pass them as hints when you connect:
Each turn reports the languages detected in the most recent audio, with or without hints. This Hindi sentence uses the English words “order”, “tracking”, and “help”:
ink-2 supports English, French, Hindi, Japanese, and Spanish, and detects them automatically. Language hints and detected languages are available on ink-preview only.

Supported languages

Use these codes for language hints. Detected languages use the same codes.

Language hints

Language hints bias the model toward the languages you expect. The model can still transcribe other languages, and without hints it detects the language automatically. Hints can improve transcription accuracy, so we recommend adding them whenever you know the languages in advance.
  • Pass one language_hint query parameter per language, up to 6. In the URL, language_hints also works. In a config message, use language_hints. A language_hint field there is ignored.
  • Use the base codes from the table above, such as pt, not regional codes such as pt-BR.
  • If you connect with an unsupported code or more than 6 hints, you get an error event and the connection closes.
  • ink-2 doesn’t take hints. It returns an error event if you pass them.
  • With ink-2 and ink-preview on Realtime STT (Manual), leave out the language query parameter. Only ink-whisper uses it. If you pass it anyway, the value must be a language ink-whisper supports.

Change hints mid-stream

With ink-preview, send a config message to replace the hints without reconnecting. This works on both realtime endpoints:
  • The new list replaces the old one. Send [] to clear it, or leave the field out to keep the current list.
  • One message can change keyterms and language_hints together. On Realtime STT (Auto), it can also change turn settings, on any model.
  • The change takes effect right away, so it also applies to words not returned yet. Those can include speech from just before the message.
  • The message is checked against the same limits as at connect time. There’s no reply when it works. If anything is wrong, you get an error event, nothing changes, and the connection stays open.
To change keyterms mid-stream, see Keyterm prompting.

Detected languages

On Realtime STT (Auto), turn.eager_end and turn.end include a languages list: the languages detected in the most recent audio. The language list is sorted by confidence, so the first entry is the language the model is most sure of. Use it when you need one language, for example to choose which language your agent replies in.  The field can be missing, such as on ink-2 or when a turn has too little audio to detect a language, so check for it first:
The list holds the languages the model is confident about. If it isn’t confident about any, it holds only the most likely one. Each language gets its own score, so the scores don’t add up to 1. For example, a turn that code-switches across Hindi and English can score high on both. On Realtime STT (Manual), there are no turn events. Instead, flush_done lists languages at each finalize, and done lists them when the session ends. Both look at recent audio, so they can include speech from before the previous finalize. If no audio arrived since the previous finalize, the event has no languages. Silence counts as audio too, and with no speech the list can name languages nobody spoke.

Good to know

  • turn.eager_end reports languages before the turn is over, and turn.end reports them again. The two can differ. If you start a reply on turn.eager_end, check the language again at turn.end, and regenerate the reply if it changed.
  • Hints don’t limit detection to the hinted languages. A turn hinted hi can come back as en if the caller spoke English.

Where to go next

Word timestamps

Start and end times for every word

Keyterm prompting

Improve accuracy on names and rare words

API reference

Every event and field