SSE streaming
Read text incrementally and handle completion, empty chunks, and interrupted connections.
Prepare
Choose a model and line supporting streaming. Install the Python dependency and set IROUTER_API_KEY and IROUTER_MODEL using the OpenAI SDK guide. This example sets the iRouter base URL directly.
Read Chat Completions deltas
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["IROUTER_API_KEY"],
base_url="https://gateway.irouter.io/v1", max_retries=0)
finished = False
with client.chat.completions.create(
model=os.environ["IROUTER_MODEL"],
messages=[{"role": "user", "content": "Explain an API in two sentences."}],
stream=True,
) as stream:
for chunk in stream:
if not chunk.choices:
continue
choice = chunk.choices[0]
print(choice.delta.content or "", end="", flush=True)
if choice.finish_reason is not None:
finished = True
print("\nFinish reason:", choice.finish_reason)
if not finished:
raise RuntimeError("Stream ended without a completion reason")delta.content is incremental text, not the complete answer. Some chunks carry roles, tool fragments, or usage, and choices can be empty. Not every chunk contains text.
Protocol differences
Protocol | Handle |
|---|---|
Chat Completions | |
Responses | Event |
Messages | |
Gemini | Content increments from |
Raw HTTP clients must split SSE events at blank lines and buffer data across network chunks. A network read is not a JSON event boundary. Use curl -N for inspection and prefer a protocol SDK in applications.
Interruptions and usage
HTTP 200 establishes the stream, not successful generation. If it ends early, preserve partial text and mark it incomplete. Do not concatenate a retried answer onto it.
Usage need not appear in every event. Enable Chat Completions stream_options.include_usage only when supported, handling usage chunks with empty choices. Missing usage does not mean no charge; reconcile in the console.