Streaming
Server-sent events on each protocol, usage and cost at the end of a stream, errors mid-stream, and cancellation.
Every chat model streams on all four protocols. Set stream: true on Chat Completions, Responses, or Messages, or call :streamGenerateContent on Gemini. The response is a stream of server-sent events in the format of the protocol you called. Streams end with token usage (on Chat Completions, when you ask for it), and Chat Completions, Responses, and Messages streams also carry the request's charge.
| Protocol | Events | Last event |
|---|---|---|
| Chat Completions | data: {chat.completion.chunk} | data: [DONE] |
| Responses | response.created, response.output_text.delta, … | response.completed, response.incomplete, or response.failed |
| Messages | message_start, content_block_delta, ping, … | message_stop |
| Gemini | data: {GenerateContentResponse} | the chunk carrying usageMetadata |
With an SDK
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.ohmygpt.com/v1",
api_key=os.environ["OHMYGPT_API_KEY"],
)
stream = client.chat.completions.create(
model="ohmygpt/mock",
messages=[{"role": "user", "content": "ping"}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)As on OpenAI, a Chat Completions stream reports usage only when you set "stream_options": {"include_usage": true}. Usage then arrives in one last chunk whose choices array is empty, so check choices before you index into it.
Raw frames
The end of a Chat Completions stream with stream_options.include_usage set:
data: {"id":"chatcmpl-ohmygpt-mock","object":"chat.completion.chunk","created":1793318400,"model":"ohmygpt/mock","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-ohmygpt-mock","object":"chat.completion.chunk","created":1793318400,"model":"ohmygpt/mock","choices":[],"usage":{"prompt_tokens":29,"completion_tokens":17,"total_tokens":46}}
: x-cost-credits: 0
data: [DONE]The end of a Messages stream:
event: message_delta
data: {"delta":{"stop_reason":"end_turn","stop_sequence":null,"stop_details":null},"type":"message_delta","usage":{"input_tokens":33,"output_tokens":17}}
: x-cost-credits: 0
event: message_stop
data: {"type":"message_stop"}If you parse the stream yourself, read it line by line, skip lines that start with :, and parse the JSON after data: . On Chat Completions, stop at data: [DONE].
Keep-alive
A model can go quiet for minutes: before its first token while it thinks, or while it builds a large tool call. Proxies and NAT devices between you and the API often close a connection that carries no bytes. To keep it open, Chat Completions, Responses, and Messages streams carry an SSE comment line after every 15 seconds of silence:
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
: keep-alive
data: {"id":"chatcmpl-EjizVUFmGLjgYzu4TcCL30VobpoVy","object":"chat.completion.chunk","created":1794566413,"model":"openai/gpt-5.1",...}SDKs skip comment lines. Chat Completions requests that send Go's default User-Agent (Go-http-client/…, as go-openai does) get none, because go-openai can't read an error that follows a comment line.
Headers that arrive in the stream
Four kinds of response headers are sent with the first output: the TEE proof receipt (X-VeriRelay-*), X-Output-Tokens-Capped when a low balance lowered the output limit, X-Parameters-Dropped when sampling parameters were left out, and Deprecation / Sunset for a model being retired. A stream that has sent a keep-alive line has already sent its headers, so these values then arrive as lowercase comment lines ahead of the first event instead:
: keep-alive
: x-output-tokens-capped: 12000
: x-verirelay-request-id: 36a420474e713c4ecbe0a6d5dfd79ed0
: x-verirelay-store-id: cFMrwmc4IeTQT14_R9y6gg
data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...}A stream that sent no keep-alive line before its first output carries them as response headers, as a non-streaming response does. X-VeriRelay-Store-Key is never written into a stream, because stream bodies are often forwarded or logged as they are: when it did not arrive as a header, look the request up to get it.
Cost of a stream
Chat Completions, Responses, and Messages streams carry the charge in Credits as an SSE comment line, : x-cost-credits: N, right before the last event. SDKs skip comment lines, so read the raw stream to capture it.
For any request, streaming or not and on any protocol, you can also look the charge up with its X-Request-ID response header:
curl https://api.ohmygpt.com/v1/generations/$REQUEST_ID \
-H "Authorization: Bearer $OHMYGPT_API_KEY"{
"object": "request_receipt",
"id": "01a19b55-f2f8-787b-941c-7e3e0bda2b0a",
"status": "succeeded",
"model": { "id": "google/gemini-3.1-flash-lite" },
"usage": { "input_tokens": 16, "output_tokens": 3, "reasoning_tokens": 118, "cache_read_tokens": 0, "cache_creation_tokens": 0 },
"cost": { "list_credits": 48, "credits": 43 }
}Errors mid-stream
Errors before the first byte come back as a normal HTTP error. Once the stream has started with 200, whether with output or with a keep-alive line, a failure ends the stream with one error event in your protocol's format:
| Protocol | Final event |
|---|---|
| Chat Completions | data: {"error": {...}} |
| Responses | event: response.failed, with response.error.code and response.error.message. When nothing came before it but keep-alive lines, a response.created event comes first. |
| Messages | event: error, with {"type": "error", "error": {...}} |
| Gemini | data: {"error": {"code": ..., "status": ...}} |
Any output received before the error is incomplete. Retry the request.
Cancellation
Close the connection to stop receiving a stream. To bound what a response can cost, set an output limit (max_tokens, max_completion_tokens or max_output_tokens) on the request.
Still stuck? Live chat · [email protected] · Telegram group