WebSocket

Configures the session's audio. This must be the first message sent. The server closes the connection if any other event arrives first, or if no event arrives within 10 seconds. Unknown fields are rejected.

Streams user audio to the agent. Send chunks continuously (for example every 20–100 ms) for the best latency. audio must be base64-encoded audio in the format declared in session_create.audio.input_format. Wait for session_ready before sending audio; audio sent earlier is dropped.

Sends a DTMF (dual-tone multi-frequency) digit to the agent.

Answers a client_tool_call whose expects_response is true. tool_call_id must match the call being answered. The decoded result may be at most 4 KiB; larger results are replaced with a result_too_large error result. Late, duplicate, or mismatched results are ignored.

Sent once after session_create, when the agent pipeline can accept audio. Reports the call record created for this session and the agent version the session is pinned to. Use call_id with the calls API to fetch the recording and transcript after the call.

The agent's speech. audio is base64-encoded audio in the format declared in session_create.audio.input_format.

Sent when the user starts speaking. Discard buffered agent audio and stop playback. The event can arrive while no agent audio is playing, in which case there is nothing to clear.

A DTMF digit the agent sends, for clients bridging a telephony system.

The agent asks this client to run a client tool. When expects_response is true, answer with a client_tool_result carrying the same tool_call_id; the invocation stays open until a result, error, or timeout. When false, dispatching the action completes the invocation.

Sent when the user starts speaking or the agent starts responding. turn comes from a single counter shared by both roles, starting at 1 and strictly increasing over the call, so turn alone identifies a turn.

Sent as the agent speaks, carrying the text spoken since the previous delta, typically one word at a time. Deltas include separator spaces, so build the running text by appending text verbatim. The user's speech is not streamed incrementally. It arrives as finalized text in turn_ended.

Sent when a turn finishes, with the complete final text for the turn. This is the version to store and display. The events for a single turn always arrive in order: turn_started, then any turn_output_text_delta events, then turn_ended. If the agent hangs up mid-turn, the connection closes without a final turn_ended; treat the close as ending any open turn.

Reports a problem with the session or an event you sent. When fatal is false, the offending event was dropped and the stream stays open. When fatal is true, the server closes the connection: code 1008 for client and protocol errors, 1011 for agent pipeline failures.