Skip to main content
Every worker shares a single NATS server by default. Declaring a pool gives a set of workers its own NATS server, so losing one server costs you only the workers attached to it.
Pool names are fixed: default, stt, tts, and agent. The default pool always exists — declare it only to override its settings. Every other declared pool gets its own nats-server-<name> Deployment and Service. A worker that references an undeclared pool fails helm upgrade at render time, naming the pool to declare.

Pool fields

Changing pool topology, transport, or output connections rolls the API automatically on helm upgrade. Expect one API rollout on the upgrade that introduces pools.

Sharding workers across pools

Splitting your workers across two pools keeps the deployment serving when a NATS pod crashes: requests continue on the pool that is still up. You are running on roughly half the capacity until the pod recovers, so expect higher latency and some queuing. For TTS, the API distributes requests across the tts and default pools in proportion to each pool’s available worker capacity, so both sets share load in normal operation. This requires release tag sonic-20260713 or later; on earlier tags the tts pool is primary and default is the fallback. STT and agent pools always treat the dedicated pool as primary and default as the fallback.

Transport

Pools carry per-task streaming I/O over NATS by default. Set transport: grpc to move that traffic onto a direct worker-to-API gRPC connection on port 50051, which improves TTFA and RTF; NATS still carries task assignment and heartbeats. Set it on every pool you declare.

Terraform

The Terraform modules mirror the chart schema with snake_case field names. Any worker that sets natsPool needs a matching nats_pools entry.
Leave natsPool unset to use the default pool — setting it to "default" is rejected.