Streaming reliability

Why stream errors are logged by default, how to keep a reverse proxy from buffering SSE, and how to prove a deployment streams incrementally with streamHealthCheck.

“Streaming works on localhost but arrives as one blob in production” is the most common deployment failure for SSE chat backends. This page covers the three things that keep token streaming healthy in production: errors that don’t vanish, proxies that don’t buffer, and a probe that proves it.

For render throttling and tool-loop bounds, see Streaming.

Errors are logged by default

The AI SDK swallows stream errors so a mid-stream failure doesn’t crash the server — which is exactly how broken production deploys end up with silent, empty logs. The handler surfaces them by default: it logs stream errors to the server console independently of onError.

export const { GET, POST, DELETE } = createChatHandler({
  getUserId: getChatUserId,
  model: anthropic('claude-sonnet-4-5'),
  // onError only maps the USER-FACING message; it does NOT silence the log.
  onError: (error) => 'Something went wrong. Please try again.',
  // logErrors defaults to true — the server-side log fires regardless of onError.
});

logErrorsboolean

Default: true

Log stream errors to the server console. Independent of onError (which only maps the user-facing copy). Set false only if you forward errors elsewhere and want to suppress the built-in log.

onError(error: unknown) => string

Map a stream error to the user-facing string the widget shows. Providing it does not silence the server log — a production failure (bad key, rate limit, wrong URL) never disappears into empty logs, the #1 documented streaming pitfall.

Anti-buffering

A genuinely streamed SSE response sends many small chunks over time. A reverse proxy or CDN that buffers the response collapses that into one late blob — the answer appears all-at-once, and it is invisible until a human notices. The handler sets the response headers that tell intermediaries not to buffer (e.g. disabling proxy buffering and chunk transformation), but a proxy you control can still override them.

Proving it: `streamHealthCheck`

streamHealthCheck makes buffering testable. Point it at your deployed chat route and it reports whether the response actually arrived incrementally (many chunks over time) or as a single late blob (buffered). Run it in CI against a staging deploy, or as a post-release smoke test, so a buffering misconfiguration fails the pipeline instead of reaching users. It validates the transport, not model correctness — it only cares that bytes stream back over time.

import { streamHealthCheck } from '@mordn/chat-widget/server';

const result = await streamHealthCheck({
  url: 'https://staging.example.com/api/chat',
  headers: { Cookie: process.env.SESSION_COOKIE! }, // must satisfy your getUserId
});

if (!result.ok) {
  console.error(result.note);
  process.exit(1); // fail CI on a buffered/broken stream
}
console.log(`Streamed ${result.chunks} chunks; first byte at ${result.firstChunkMs}ms.`);

Options

urlstringrequired

Absolute URL of the mounted chat route’s POST endpoint (e.g. https://staging.example.com/api/chat).

headersRecord<string, string>

Headers used to authenticate the probe. Your getUserId must accept these (a session cookie or a test bearer token). Without valid auth the endpoint returns 401 and the probe reports ok: false.

messagestring

Default: "ping"

Probe message text.

conversationIdstring

Conversation id for the probe turn. Defaults to an ephemeral health-* id.

timeoutMsnumber

Default: 20000

Abort the probe after this many ms.

fetchImpltypeof fetch

Custom fetch implementation (defaults to global fetch).

Result

interface StreamHealthResult {
  ok: boolean;              // true when it streamed incrementally with a 2xx
  status: number;           // HTTP status (0 if the request never completed)
  chunks: number;           // non-empty network reads observed
  firstChunkMs: number | null; // ms to the first non-empty chunk
  totalMs: number;          // ms to stream end
  likelyBuffered: boolean;  // heuristic: arrived as ~one late chunk
  note: string;             // human-readable summary / remediation pointer
}

streamHealthCheck never throws — failures come back as { ok: false, note }. The likelyBuffered heuristic flags a response that arrived as a single chunk, or whose first chunk landed right as the stream ended — both strong signs an intermediary buffered the body.

See Streaming for throttling, stopWhen, and the onChatFinish telemetry hook.