WebSocket vs. HTTP Streaming for Real-Time TTS Delivery

Plain HTTP, the REST model most engineers learn first, is a closed loop. The client sends the full text, the server renders the full audio file, and only then does anything come back. No sample plays until synthesis finishes, whether the client is a browser fetch call, a mobile SDK, or a backend job. Easy to wire up, and no special client logic needed. But it can't do real-time voice, structurally, and waiting for a whole file before the first word plays creates the kind of dead air that stalls a conversation.

HTTP streaming loosens that up a bit. The client still sends one complete request, but the server streams audio back as it's generated, so playback starts mid-synthesis instead of after it. Under the hood this rides on HTTP/1.1 persistent connections and chunked transfer encoding, where the server sends data in pieces without declaring the total length up front. The catch: it only flows one way, server to client. Once the request goes out, the client can't send anything new; it has to open a new connection to say anything else. There's also a trap that catches people who haven't hit it before: reverse proxies like Nginx buffer responses by default, so chunked or SSE events arrive in bursts instead of a steady trickle, unless someone remembers to switch proxy buffering off.

Server-Sent Events sit on top of HTTP streaming as a specific format, sometimes described as one request with an "infinite" response. SSE is text-only by nature, so binary audio has to be Base64-encoded first, which adds roughly a third to the payload size. That's a real tax when the payload is audio and not a chat message. SSE does sit on standard HTTP, which gives it some resilience when a connection drops mid-stream.

WebSocket works differently at the root. It's a full-duplex, persistent channel over one TCP connection, so both sides send whenever they want without waiting on a request-response cycle. It starts life as an HTTP request too, but that request asks to upgrade: the client signals it wants WebSocket, the server agrees, and the protocol switches over in a single handshake before the channel stays open. Audio travels in native binary frames, skipping the encoding tax SSE's Base64 approach carries. What matters most for TTS specifically is the two-way part: the client can send new text, an interruption signal, or a control message at any point mid-stream, without touching the connection.

The per-chunk overhead gap that makes latency budgets tight

Here's where the mechanics turn into milliseconds. Every HTTP or REST request drags along hundreds to thousands of bytes of headers, plus a fresh TCP handshake if the connection isn't kept alive, while a WebSocket frame, once the handshake is done, carries somewhere between 2 and 14 bytes of framing overhead. That gap barely matters for a single request, but it matters a lot for streaming audio, where chunks show up every 50 to 250 milliseconds and HTTP keeps paying that header cost over and over.

One provider measurement puts this in concrete terms: HTTPS POST delivery landed at 258ms P50 and 274ms P95, while switching to WebSocket with connection reuse dropped P50 to 214ms. Forty-four milliseconds doesn't sound like much on paper, but against a 300ms total budget, it's the difference between a pause that reads as the system thinking and one that reads as the system stalling.

There's a second effect stacked on top of the framing savings, and it's the one people miss. Because WebSocket runs both ways at once, the server starts synthesizing audio the moment the first text tokens land, rather than waiting for the whole sentence or the whole request to show up. HTTP streaming's one-way, request-first model limits how early synthesis can begin. So the two savings stack: lower per-chunk overhead and an earlier synthesis start both push toward staying under that sub-300ms ceiling, and losing either one eats into the same shrinking budget.

Why TTFB benchmarks mislead and what to measure instead

Most published TTS benchmarks report Time To First Byte, the delay until the server sends anything at all. Here's the trap: the first bytes out of a streaming TTS API are usually container metadata, a WAV header, an Ogg identification page, an MP3 ID3 tag, not a single playable sample. A server fires that metadata back fast and still leaves the user waiting a while before actual sound reaches their speakers.

A benchmark can therefore report a fast TTFB while the person on the other end experiences something closer to a stall. The metric that actually tracks with what someone hears is Time To First Audio, TTFA: the gap between submitting a request and the first chunk of playable audio arriving. Anyone comparing providers, or tuning a pipeline they already run, should measure TTFA instead of TTFB, because the two numbers can diverge by a wide margin depending on which codec and container format is doing the wrapping.

That's not just a footnote about measurement. It flips which protocol looks better in a given benchmark. WebSocket's low framing overhead shows up clearly in TTFA, because it speeds up the actual audio delivery and not just the header that arrives first. Measure TTFB alone and the advantage disappears from view entirely, an easy thing for an industry this focused on milliseconds to overlook.

Where current providers actually land on latency and protocol choice

The spread across vendors as of mid-2026 shows how much protocol and model architecture both shape the floor. Transport sets the ceiling, but the model decides whether anyone gets anywhere near it. OpenAI's Realtime TTS-2 reports P99 under 100ms, and the Flash variant gets that under 50ms, both running over WebSocket with no buffering delay in between. ElevenLabs' Flash v2.5 sits around 75ms of model inference latency. Cartesia's Sonic-3.5 reports roughly 82ms TTFA, with sub-150ms as an explicit design target, and WebSocket support means audio starts arriving almost as soon as text goes out. Deepgram's Aura-2, by contrast, runs around 313ms, above the ITU-T outer bound of 300ms, despite running the same class of transport as everyone else on this list. Same protocol family, worse result, because the bottleneck moved somewhere else in the stack.

Protocol decisions across these companies follow a consistent logic. ElevenLabs runs two separate service tracks: a WebSocket-based one built for real-time work with word-level timestamps and interruption handling, and an HTTP-based one for batch synthesis where nothing is waiting on the other end. Deepgram has said outright that Aura's WebSocket interface beats sentence-chunked REST calls on response time, and the gap widens as the LLM's response gets longer. Smallest AI runs Lightning v3.1 across HTTP, SSE, and WebSocket endpoints, with servers geo-routed across the US and India, which makes a separate point worth holding onto: regional routing and protocol choice are two different levers, not one and the same.

Zoom out and the industry has more or less converged on WebSocket for anything interactive or conversational, HTTP streaming for anything simpler or batch. The per-chunk savings and two-way signaling become decisive the moment a human is sitting there waiting for a reply. Put the whole voice pipeline together and the pressure compounds: speech-to-text eating 200 to 400ms, the LLM's first token landing somewhere between 100 and 500ms, a fast streaming TTS endpoint adding 100 to 300ms on top. Staying under 700ms end to end takes tuning at every layer, and transport is one of the few knobs an engineer can actually turn without waiting on somebody else's model update.

The scalability liabilities that WebSocket's latency advantage carries with it

None of this comes free, and here's the part that gets skipped when the advice is just "use WebSocket everywhere": that advice falls apart for most of a typical product's traffic. A persistent, stateful connection is cheap to demo and expensive to run at scale. Every open WebSocket connection holds memory, a file descriptor, and server resources for as long as it stays alive, and that cost scales linearly with concurrent connections in a way HTTP's stateless model never has to.

Load balancing is where this gets genuinely difficult. HTTP requests land on any available server, no strings attached, since nothing needs to persist between calls. WebSocket connections need session stickiness: the connection has to stay pinned to the same backend node for its entire life. Standard round-robin or least-connections balancing doesn't work here without extra setup, because once a session is established, every message for it has to keep flowing through that one server. Two users on different nodes who need to exchange state now have a cross-server messaging problem that didn't exist five minutes ago.

The usual fixes come with their own weight. Distributed caches like Redis or Memcached track which user sits on which node so requests route correctly: one more system to keep alive, one more thing that fails quietly at the worst time. Pub/sub layers, Redis Pub/Sub, Kafka, NATS, broadcast messages to every relevant node so a client gets what it needs regardless of which server holds its connection. That's another piece of infrastructure someone has to run and get paged for when it breaks.

Security is its own concern. WebSocket connections slip past traditional firewall rules built for request-response traffic, they're exposed to man-in-the-middle attacks and cross-site WebSocket hijacking, and they're flatly incompatible with some proxies, firewalls, and antivirus software, each needing its own workaround. HTTP streaming, being stateless, dodges most of this: CDNs, reverse proxies, and standard load balancers handle it with no special setup, which is why SSE tends to be the easier thing to scale. The latency win WebSocket offers is real, but it's rented, not owned, and the rent gets paid in engineering hours. The commercial side of this is concrete with providers like ElevenLabs, where connection count isn't just an infrastructure line item — it's a pricing conversation too.

Connection patterns and use-case types that favor each protocol

So when does WebSocket actually earn its keep? Four situations stand out. The application is conversational with real turn-taking, so the two-way channel lets someone interrupt or send a new utterance mid-stream without tearing down the connection. The latency budget is tight enough that TTS has to stay under roughly 200ms to leave room for STT and the LLM elsewhere in the chain. The pipeline needs input streaming, meaning synthesis has to start before the LLM has even finished generating its full response, something only WebSocket supports. Session continuity also matters: word-level timestamps, interruption handling, ongoing audio context, all needing a channel that stays open and stateful.

HTTP streaming earns its keep in the opposite situations, and it's worth saying plainly: reaching for WebSocket by default, for anything that isn't a live back-and-forth, is over-engineering dressed up as diligence. Narration, content read aloud on demand, assistive reading tools, notification audio: these are one-shot exchanges with no back-channel needed at all. Infrastructure simplicity matters more than shaving off forty milliseconds here, so standard load balancers just work without sticky sessions or a pub/sub layer bolted on. Some corporate networks and CDN setups actively block or degrade WebSocket traffic, while HTTP streaming passes through without anyone noticing. And when a team wants a fallback for when WebSocket connections can't be guaranteed, HTTP streaming is the sane backup.

SSE fits a narrower lane still: text-heavy event feeds where audio is a secondary feature at best. The Base64 penalty rules it out as a serious primary transport for audio at any real scale.

The decision, boiled down, starts with two questions: how long does the connection need to live, and does data need to move both ways? Answer yes to bidirectionality and input streaming, and WebSocket is the answer regardless of how much scaling complexity comes with it. Answer no, one-shot and one-directional, and HTTP streaming's simpler operational profile wins. Otherwise, a team ends up paying for Redis and Kafka to solve a problem that never existed in the first place.

How geographic distance and network path interact with protocol choice

Protocol is one variable in the latency equation, and network path is a multiplier that swamps it entirely; no transport decision fixes bad geography. A user in Singapore talking to infrastructure sitting in US-East is looking at 250-plus milliseconds of round-trip latency before a single byte of processing has even happened.

Real internet paths add routing hops, peering exchanges, submarine cable crossings, the physical reality of how packets actually travel. Route a WebSocket connection through a distant cloud region and transport alone tacks on 200ms in the tail latency. Compare that to a well-configured WebRTC setup, which runs over UDP instead of TCP: transport there adds something closer to 20 to 40ms round-trip. That gap means WebSocket, for all its advantages over HTTP streaming, is not automatically the last word on latency for the most demanding deployments.

The practical fix is a better map, not a better protocol. Geo-routed endpoints, the same approach Smallest AI takes across its US and India regions, cut network latency independently of whatever protocol sits on top. WebSocket's persistent connection amortizes its handshake cost across a whole session, a real saving, but only for that session; a user reconnecting from a different region pays the handshake fresh, every single time. For any voice deployment with a global user base, regional edge nodes are load-bearing architecture, not a nice-to-have tuning knob. Tune the protocol all anyone wants; skip the regional distribution, and there's still latency sitting on the table, unclaimed and expensive.

Avatar and lip-sync pipelines where transport choice affects more than audio latency

Layer video onto this, an animated avatar or a lip-sync agent, and transport choice stops being just an audio-latency question. Now it decides whether the mouth movements match the words at all. In a pipeline running ASR into an LLM into TTS into a rendering engine, audio and video have to land in sync, and any transport that introduces uneven or bursty delivery, the kind of thing a buffered reverse proxy does to chunked HTTP responses, risks pulling the two streams apart. A WebSocket channel delivering audio frames at a steady cadence gives a lip-sync engine something predictable to align against. A connection delivering audio in irregular bursts gives it a moving target instead, and a moving target is exactly what makes an avatar's mouth drift out of step with its own voice. The transport decision made earlier in this piece for reasons of pure audio latency turns out to matter just as much once there's a face attached to the voice. Speed of arrival was only ever half the question; what happens once the words land, in sync or not, is the other half.