> ## Documentation Index > Fetch the complete documentation index at: https://docs.cartesia.ai/llms.txt > Use this file to discover all available pages before exploring further. # Agents (WebSocket) > Stream audio between your application and a hosted Line voice agent over a single bidirectional WebSocket. Use this for web apps, mobile apps, or to bridge your own telephony provider into a Line agent. All messages are JSON text frames. Audio is exchanged as base64-encoded payloads inside those messages, not as binary frames. **Basic Usage**: 1. Connect to `wss://api.cartesia.ai/agents/stream/{agent_id}` with an `Authorization: Bearer ` header (or `access_token` query parameter in the browser). Generate an agent access token via the [`/access-token` endpoint](/api-reference/auth/access-token#body-grants-agent). 2. Send a `start` event as the first message — the server closes the connection if any other event arrives first. 3. Wait for the server's `ack`, which echoes the resolved config and returns a `stream_id` if you didn't provide one. 4. Stream user audio as `media_input` events (base64-encoded, in the format declared in `start.config.input_format`). 5. Receive agent audio as `media_output` events. Handle `clear` (interrupt) and `transfer_call` (telephony handoff) events as they arrive. 6. Send standard WebSocket ping frames every 60–90 seconds to avoid the 180-second inactivity timeout. For a higher-level walkthrough and a runnable browser example, see the [WebSocket API guide](/line/integrations/websocket-api). ## AsyncAPI ````yaml asyncapi.yml /agents/stream/{agent_id} id: /agents/stream/{agent_id} title: /agents/stream/{agent_id} description: > Stream audio between your application and a hosted Line voice agent over a single bidirectional WebSocket. Use this for web apps, mobile apps, or to bridge your own telephony provider into a Line agent. All messages are JSON text frames. Audio is exchanged as base64-encoded payloads inside those messages, not as binary frames. **Basic Usage**: 1. Connect to `wss://api.cartesia.ai/agents/stream/{agent_id}` with an `Authorization: Bearer ` header (or `access_token` query parameter in the browser). Generate an agent access token via the [`/access-token` endpoint](/api-reference/auth/access-token#body-grants-agent). 2. Send a `start` event as the first message — the server closes the connection if any other event arrives first. 3. Wait for the server's `ack`, which echoes the resolved config and returns a `stream_id` if you didn't provide one. 4. Stream user audio as `media_input` events (base64-encoded, in the format declared in `start.config.input_format`). 5. Receive agent audio as `media_output` events. Handle `clear` (interrupt) and `transfer_call` (telephony handoff) events as they arrive. 6. Send standard WebSocket ping frames every 60–90 seconds to avoid the 180-second inactivity timeout. For a higher-level walkthrough and a runnable browser example, see the [WebSocket API guide](/line/integrations/websocket-api). servers: - id: production protocol: wss host: api.cartesia.ai bindings: [] variables: [] address: /agents/stream/{agent_id} parameters: - id: agent_id jsonSchema: type: string description: | ID of the deployed Line agent to connect to. description: | ID of the deployed Line agent to connect to. type: string required: true deprecated: false - id: cartesia_version jsonSchema: type: string description: API version, e.g. `2026-03-01` description: API version, e.g. `2026-03-01` type: string required: true deprecated: false bindings: [] operations: - &ref_1 id: sendAgentsEvents title: Send agents events description: > Client → server events. Send `start` first to initialize the stream, then stream user audio as `media_input` events. `dtmf` and `custom` events can be sent at any point after the initial `ack`. type: receive messages: - &ref_3 id: agentsStartEvent payload: - name: Start Event description: > Initializes the stream. **This must be the first message sent** — the server closes the connection if any other event arrives first. type: object properties: - name: event type: string description: Event type identifier. enumValues: - start required: true - name: stream_id type: string description: >- Stream identifier. If omitted, the server generates one and returns it in the `ack` event. required: false - name: config type: object description: >- Stream configuration. Overrides the agent's defaults for this call. required: false properties: - name: input_format type: string description: Audio format for client-sent audio. enumValues: - mulaw_8000 - pcm_16000 - pcm_24000 - pcm_44100 required: false - name: output_audio_delivery type: string description: > Controls how the server delivers agent audio. `speaking_pace` (default) paces audio to playback speed. `as_available` sends audio as fast as the model produces it, so a client that buffers locally can play it back on its own clock for lower latency; on interruption the server sends a `clear` event and the client discards its unplayed buffer. Cannot be used when the agent is configured with background audio — the connection is rejected in that case. enumValues: - speaking_pace - as_available required: false - name: voice_id type: string description: Overrides the agent's default TTS voice. required: false - name: agent type: object description: >- Per-call overrides for the agent. Useful for previewing changes without redeploying. required: false properties: - name: introduction type: string description: Overrides the agent's introduction for this call. required: false - name: system_prompt type: string description: Overrides the agent's system prompt for this call. required: false - name: metadata type: object description: > Custom metadata passed through to the agent. The `to` and `from` keys are reserved — `to` is the destination identifier for call routing (defaults to the agent ID) and `from` is the source identifier (defaults to `websocket`). required: false headers: [] jsonPayloadSchema: type: object required: - event properties: event: type: string enum: - start description: Event type identifier. x-parser-schema-id: stream_id: type: string description: >- Stream identifier. If omitted, the server generates one and returns it in the `ack` event. x-parser-schema-id: config: type: object description: >- Stream configuration. Overrides the agent's defaults for this call. properties: input_format: type: string enum: - mulaw_8000 - pcm_16000 - pcm_24000 - pcm_44100 description: Audio format for client-sent audio. x-parser-schema-id: output_audio_delivery: type: string enum: - speaking_pace - as_available default: speaking_pace description: > Controls how the server delivers agent audio. `speaking_pace` (default) paces audio to playback speed. `as_available` sends audio as fast as the model produces it, so a client that buffers locally can play it back on its own clock for lower latency; on interruption the server sends a `clear` event and the client discards its unplayed buffer. Cannot be used when the agent is configured with background audio — the connection is rejected in that case. x-parser-schema-id: voice_id: type: string description: Overrides the agent's default TTS voice. x-parser-schema-id: x-parser-schema-id: agent: type: object description: >- Per-call overrides for the agent. Useful for previewing changes without redeploying. properties: introduction: type: string description: Overrides the agent's introduction for this call. x-parser-schema-id: system_prompt: type: string description: Overrides the agent's system prompt for this call. x-parser-schema-id: x-parser-schema-id: metadata: type: object description: > Custom metadata passed through to the agent. The `to` and `from` keys are reserved — `to` is the destination identifier for call routing (defaults to the agent ID) and `from` is the source identifier (defaults to `websocket`). additionalProperties: true x-parser-schema-id: x-parser-schema-id: AgentsStartEvent title: Start Event description: > Initializes the stream. **This must be the first message sent** — the server closes the connection if any other event arrives first. example: |- { "event": "start", "stream_id": "unique_id", "config": { "input_format": "pcm_44100", "output_audio_delivery": "as_available", "voice_id": "a0e99841-438c-4a64-b679-ae501e7d6091" }, "agent": { "introduction": "Hello, I'm an AI assistant", "system_prompt": "### Your Role \n You are a helpful assistant" }, "metadata": { "to": "user@example.com", "from": "+1234567890" } } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsStartEvent - &ref_4 id: agentsMediaInputEvent payload: - name: Media Input Event description: > Streams user audio to the agent. Send chunks continuously (for example every 20–100 ms) for the best latency. `media.payload` must be base64-encoded audio in the format declared in `start.config.input_format`. type: object properties: - name: event type: string description: Event type identifier. enumValues: - media_input required: true - name: stream_id type: string description: Stream identifier returned in the `ack` event. required: false - name: media type: object required: true properties: - name: payload type: string description: >- Base64-encoded audio in the format specified by `start.config.input_format`. required: true headers: [] jsonPayloadSchema: type: object required: - event - media properties: event: type: string enum: - media_input description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier returned in the `ack` event. x-parser-schema-id: media: type: object required: - payload properties: payload: type: string description: >- Base64-encoded audio in the format specified by `start.config.input_format`. x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: AgentsMediaInputEvent title: Media Input Event description: > Streams user audio to the agent. Send chunks continuously (for example every 20–100 ms) for the best latency. `media.payload` must be base64-encoded audio in the format declared in `start.config.input_format`. example: |- { "event": "media_input", "stream_id": "unique_id", "media": { "payload": "base64_encoded_audio_data" } } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsMediaInputEvent - &ref_5 id: agentsDtmfEvent payload: - name: DTMF Event description: | Sends a DTMF (dual-tone multi-frequency) tone to the agent. type: object properties: - name: event type: string description: Event type identifier. enumValues: - dtmf required: true - name: stream_id type: string description: Stream identifier returned in the `ack` event. required: false - name: dtmf type: string description: A single DTMF digit. One of `0`–`9`, `*`, or `#`. required: true headers: [] jsonPayloadSchema: type: object required: - event - dtmf properties: event: type: string enum: - dtmf description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier returned in the `ack` event. x-parser-schema-id: dtmf: type: string description: A single DTMF digit. One of `0`–`9`, `*`, or `#`. x-parser-schema-id: x-parser-schema-id: AgentsDtmfEvent title: DTMF Event description: | Sends a DTMF (dual-tone multi-frequency) tone to the agent. example: |- { "event": "dtmf", "stream_id": "unique_id", "dtmf": "1" } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsDtmfEvent - &ref_6 id: agentsCustomEvent payload: - name: Custom Event description: > Sends arbitrary metadata to the agent mid-call. The agent receives the payload as a custom event in its handler. type: object properties: - name: event type: string description: Event type identifier. enumValues: - custom required: true - name: stream_id type: string description: Stream identifier returned in the `ack` event. required: false - name: metadata type: object description: Arbitrary key-value data to forward to the agent. required: true headers: [] jsonPayloadSchema: type: object required: - event - metadata properties: event: type: string enum: - custom description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier returned in the `ack` event. x-parser-schema-id: metadata: type: object description: Arbitrary key-value data to forward to the agent. additionalProperties: true x-parser-schema-id: x-parser-schema-id: AgentsCustomEvent title: Custom Event description: > Sends arbitrary metadata to the agent mid-call. The agent receives the payload as a custom event in its handler. example: |- { "event": "custom", "stream_id": "unique_id", "metadata": { "user_id": "user123", "session_info": "custom_data" } } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsCustomEvent bindings: [] extensions: &ref_0 - id: x-parser-unique-object-id value: /agents/stream/{agent_id} - &ref_2 id: receiveAgentsEvents title: Receive agents events description: > Server → client events. The server emits `ack` once after `start`, then streams agent audio as `media_output` events. `clear` indicates the agent is interrupting itself (e.g. when the user starts speaking). `transfer_call` asks the client to hand the call off to a phone number. type: send messages: - &ref_7 id: agentsAckEvent payload: - name: Ack Event description: > Confirms the stream configuration. Echoes back the resolved `config` and `agent` overrides, and returns a server-generated `stream_id` if one wasn't provided in `start`. type: object properties: - name: event type: string description: Event type identifier. enumValues: - ack required: true - name: stream_id type: string description: >- Stream identifier. Server-generated if not provided in `start`. required: true - name: config type: object description: Resolved stream configuration. required: false properties: - name: input_format type: string enumValues: - mulaw_8000 - pcm_16000 - pcm_24000 - pcm_44100 required: false - name: output_audio_delivery type: string enumValues: - speaking_pace - as_available required: false - name: voice_id type: string required: false - name: agent type: object description: Resolved agent overrides for this call. required: false properties: - name: introduction type: string required: false - name: system_prompt type: string required: false headers: [] jsonPayloadSchema: type: object required: - event - stream_id properties: event: type: string enum: - ack description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier. Server-generated if not provided in `start`. x-parser-schema-id: config: type: object description: Resolved stream configuration. properties: input_format: type: string enum: - mulaw_8000 - pcm_16000 - pcm_24000 - pcm_44100 x-parser-schema-id: output_audio_delivery: type: string enum: - speaking_pace - as_available x-parser-schema-id: voice_id: type: string x-parser-schema-id: x-parser-schema-id: agent: type: object description: Resolved agent overrides for this call. properties: introduction: type: string x-parser-schema-id: system_prompt: type: string x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: AgentsAckEvent title: Ack Event description: > Confirms the stream configuration. Echoes back the resolved `config` and `agent` overrides, and returns a server-generated `stream_id` if one wasn't provided in `start`. example: |- { "event": "ack", "stream_id": "unique_id", "config": { "input_format": "pcm_44100", "output_audio_delivery": "as_available", "voice_id": "a0e99841-438c-4a64-b679-ae501e7d6091" }, "agent": { "system_prompt": "### Your Role \n You are a helpful assistant", "introduction": "Hello, I'm an AI assistant" } } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsAckEvent - &ref_8 id: agentsMediaOutputEvent payload: - name: Media Output Event description: > Agent audio. `media.payload` is base64-encoded audio in the agent's configured output format. type: object properties: - name: event type: string description: Event type identifier. enumValues: - media_output required: true - name: stream_id type: string description: Stream identifier. required: true - name: media type: object required: true properties: - name: payload type: string description: >- Base64-encoded audio in the agent's configured output format. required: true headers: [] jsonPayloadSchema: type: object required: - event - stream_id - media properties: event: type: string enum: - media_output description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier. x-parser-schema-id: media: type: object required: - payload properties: payload: type: string description: >- Base64-encoded audio in the agent's configured output format. x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: AgentsMediaOutputEvent title: Media Output Event description: > Agent audio. `media.payload` is base64-encoded audio in the agent's configured output format. example: |- { "event": "media_output", "stream_id": "unique_id", "media": { "payload": "base64_encoded_audio_data" } } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsMediaOutputEvent - &ref_9 id: agentsClearEvent payload: - name: Clear Event description: > The agent is interrupting itself — for example, because the user started speaking. Stop playing any buffered audio for this stream. type: object properties: - name: event type: string description: Event type identifier. enumValues: - clear required: true - name: stream_id type: string description: Stream identifier. required: true headers: [] jsonPayloadSchema: type: object required: - event - stream_id properties: event: type: string enum: - clear description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier. x-parser-schema-id: x-parser-schema-id: AgentsClearEvent title: Clear Event description: > The agent is interrupting itself — for example, because the user started speaking. Stop playing any buffered audio for this stream. example: |- { "event": "clear", "stream_id": "unique_id" } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsClearEvent - &ref_10 id: agentsTransferCallEvent payload: - name: Transfer Call Event description: > The agent wants to transfer the call to a phone number. The client is responsible for initiating the transfer on its telephony side. type: object properties: - name: event type: string description: Event type identifier. enumValues: - transfer_call required: true - name: stream_id type: string description: Stream identifier. required: true - name: transfer type: object required: true properties: - name: target_phone_number type: string description: E.164 phone number to transfer the call to. required: true headers: [] jsonPayloadSchema: type: object required: - event - stream_id - transfer properties: event: type: string enum: - transfer_call description: Event type identifier. x-parser-schema-id: stream_id: type: string description: Stream identifier. x-parser-schema-id: transfer: type: object required: - target_phone_number properties: target_phone_number: type: string description: E.164 phone number to transfer the call to. x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: AgentsTransferCallEvent title: Transfer Call Event description: > The agent wants to transfer the call to a phone number. The client is responsible for initiating the transfer on its telephony side. example: |- { "event": "transfer_call", "stream_id": "unique_id", "transfer": { "target_phone_number": "+1234567890" } } bindings: [] extensions: - id: x-parser-unique-object-id value: agentsTransferCallEvent bindings: [] extensions: *ref_0 sendOperations: - *ref_1 receiveOperations: - *ref_2 sendMessages: - *ref_3 - *ref_4 - *ref_5 - *ref_6 receiveMessages: - *ref_7 - *ref_8 - *ref_9 - *ref_10 extensions: - id: x-parser-unique-object-id value: /agents/stream/{agent_id} securitySchemes: - id: apiKey name: X-API-Key type: httpApiKey description: Use an API key when you're calling from a trusted server. in: header extensions: [] - id: accessTokenQuery name: access_token type: httpApiKey description: >- Use a short-lived access token when calling from a browser or client app. [Learn more here](/get-started/authenticate-your-client-applications). in: query extensions: [] ````