Stream Inputs using Continuations

Learn how to stream input text to the Sonic API.

In many real-time use cases, you don’t have input text available upfront—like when you’re generating it on the fly using a language model. For these cases, Sonic supports input streaming through a feature we call continuations.

This guide will cover how input streaming works from the perspective of the Sonic model. If you just want to implement input streaming, see the WebSocket API reference, which implements continuations using contexts.

Continuations

Continuations are generations that extend already generated speech. They’re called continuations because you’re continuing the generation from where the last one left off, maintaining the prosody of the previous generation.

If you don’t use continuations, you get sudden changes in prosody that create seams in the audio.

Prosody refers to the rhythm, intonation, and stress in speech. It’s what makes speech flow naturally and sound human-like.

Let’s say we’re using an LLM and it generates a transcript in three parts, with a one second delay between each part:

  1. Hello, my name is Sonic.
  2. It's very nice
  3. to meet you.

To generate speech for the whole transcript, we might think to generate speech for each part independently and stitch the audios together:

no_continuations
Figure 1: Generate transcripts independently & stitch them together.

Unfortunately, we end up with speech that has sudden changes in prosody and strange pacing:

Now, let’s try the same transcripts, but using continuations. The setup looks like this:

continuations
Figure 2: Generate transcripts using continuations.

Here’s what we get:

As you can hear, this output sounds seamless and natural.

You can scale up continuations to any number of inputs. There is no limit.

Caveat: Streamed inputs should form a valid transcript when joined

This means that Hello, world! can be followed by How are you? (note the leading space) but not How are you?, since when joined they form the invalid transcript Hello, world!How are you?.

In practice, this means you should maintain spacing and punctuation in your streamed inputs.

Caveat: Word-by-word or token-by-token inputs

TL;DR: If you want to stream the input in word-by-word, your code should buffer the first sentence of the input upfront. Then you can stream the remaining input text word-by-word.

Let’s try the following transcripts with continuations: ['Hello, my name is Sonic.', "It's ", 'very ', 'nice ', 'to ', 'meet ', 'you.']