Fixing SSE Connection Drops Behind Reverse Proxies for LLM Chains

The first token arrives. Thirty seconds later the tab spins and the Route Handler is still running. That is not model failure. Something in front of Node closed the socket because the stream looked idle.

Buffering is a different bug. If the UI paints nothing until the full answer, read Caddy flush_interval for Next.js 15 SSE. This post is the drop in the middle of a multi-step graph.

AgenticPacks OSS streams chat with the AI SDK: Agentic-Packs/agenticpacks-oss. Long supervisor graphs, Tavily, and summarizer nodes are Pro. Caddy in front of those graphs is Pro.

Three clocks that kill a long graph

A LangGraph supervisor that calls a researcher then a summarizer often runs 30s to 3 minutes. Tokens pause between nodes. Proxies treat a quiet socket as dead.

Caddy flush_interval -1 stops coalescing. It does not extend an upstream timeout. You still need the proxy to wait.

NGINX:

location /api/ {
  proxy_pass http://127.0.0.1:3000;
  proxy_http_version 1.1;
  proxy_set_header Connection "";
  proxy_buffering off;
  proxy_cache off;
  proxy_read_timeout 360s;
  proxy_send_timeout 360s;
}

If you came from NGINX plus Certbot, the Caddy SSE post is the shorter production path. Pro ships that Caddyfile.

Keep the stream dirty

SSE is still HTTP/1.1. If no bytes move during a Tavily round-trip, the proxy may still drop you even with a high timeout.

Write a comment line on an interval while the graph is working:

: keep-alive

The AI SDK data stream and LangGraph HTTP adapters can emit these. If you rolled a raw ReadableStream, schedule a ping every 15s until controller.close().

Do not gzip the SSE route. encode gzip at the proxy for HTML is fine. For text/event-stream, skip it or you reintroduce buffering.

Cloudflare

Orange-cloud HTTP has a 100s cutoff on many plans. A 3 minute agent will die. Options:

• DNS only (grey cloud) for the API host, TLS on Caddy. That is the Compose + Caddy topology.
• A separate api. hostname that is not proxied.
• Shorter graphs so each request stays under the limit (usually the wrong product decision).

Full (strict) SSL still needs a real cert on origin. Grey-cloud until ACME succeeds, then decide whether to proxy.

App Router details

• runtime = "nodejs" for the stream route. Edge has a duration cap.
• Do not await the full graph before you return the response. Return toDataStreamResponse() (or the LangGraph HTTP response) so the client attaches while nodes run.
• Vercel serverless max duration will kill the same workload. That is why the 25k MAU cost post exists.

Tired of streams that die between LangGraph nodes?

AgenticPacks OSS is the streaming chat baseline: Agentic-Packs/agenticpacks-oss.

Pro runs the 3-node graph behind Caddy with flush_interval -1. See Caddy flush_interval on the LangGraph route in Pro.

Debug order

1. Hit the Route Handler with curl -N on the VPS, bypassing the proxy. If that stays up, the app is fine.
2. Hit through Caddy or NGINX.
3. Hit through Cloudflare.

The first hop that drops is the timeout you raise. Do not add retries that replay the whole graph. Checkpoint state instead: human-in-the-loop agents.

---

Where to Go From Here

You do not have to keep shipping a chat UI that silently dies on node two.

Option A: Free open-source baseline (MIT)

• GitHub: github.com/Agentic-Packs/agenticpacks-oss
• What is included: Next.js 15, Compose, PostgreSQL 16, Auth.js v5, AI SDK streaming chat, single-plan Stripe. No Caddy, no LangGraph.
• Cost: free, MIT.

---

Option B: Ship with AgenticPacks Pro ($149 one-time)

If the product is a supervisor graph, not a single completion:

• Automated Hetzner provisioning: Terraform, firewall, Caddy SSL with SSE flush.
• Production multi-agent core: LangGraph supervisor, Tavily researcher, summarizer, live reasoning stream.
• Usage & Unit Economics: Stripe subscriptions and token-to-dollar ledger.