Realtime
Speech-to-speech and text conversations, live transcription, and browser access with the OpenAI Realtime API.
wss://api.ohmygpt.com/v1/realtime serves the OpenAI Realtime API: one WebSocket per conversation, carrying audio and text both ways. Client and server events follow OpenAI's Realtime format, so OpenAI's SDKs and examples work once you change the base URL, the key, and the model ID.
| Endpoint | wss://api.ohmygpt.com/v1/realtime?model={model} |
| Auth | Authorization: Bearer $OHMYGPT_API_KEY, or the WebSocket subprotocol below |
| Models | openai/gpt-realtime-2.1, openai/gpt-realtime-2.1-mini, and the other models that list /v1/realtime in GET /v1/models |
import asyncio
import os
from openai import AsyncOpenAI
client = AsyncOpenAI(api_key=os.environ["OHMYGPT_API_KEY"], base_url="https://api.ohmygpt.com/v1")
async def main():
async with client.realtime.connect(model="openai/gpt-realtime-2.1-mini") as conn:
await conn.session.update(session={"type": "realtime", "output_modalities": ["text"]})
await conn.conversation.item.create(item={
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": "Say hello in one short sentence."}],
})
await conn.response.create()
async for event in conn:
if event.type == "response.output_text.delta":
print(event.delta, end="", flush=True)
elif event.type == "response.done":
print()
print(event.response.usage)
break
asyncio.run(main())Install the SDK with pip install "openai[realtime]". The script prints the reply, then the usage the response is billed on:
Hello there!
RealtimeResponseUsage(input_token_details=RealtimeResponseUsageInputTokenDetails(audio_tokens=0, cached_tokens=128, cached_tokens_details=CachedTokensDetails(audio_tokens=0, image_tokens=0, text_tokens=128), image_tokens=0, text_tokens=129), input_tokens=129, output_token_details=RealtimeResponseUsageOutputTokenDetails(audio_tokens=0, text_tokens=36, reasoning_tokens=29), output_tokens=36, total_tokens=165)Client secrets
A client secret is a short-lived token that opens Realtime sessions in place of your API key. Create it on your server, then hand it to the browser or mobile app:
curl https://api.ohmygpt.com/v1/realtime/client_secrets \
-H "Authorization: Bearer $OHMYGPT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"expires_after": {"anchor": "created_at", "seconds": 600},
"session": {
"type": "realtime",
"model": "openai/gpt-realtime-2.1-mini",
"instructions": "Be brief."
}
}'{
"expires_at": 1793318400,
"session": {
"instructions": "Be brief.",
"model": "openai/gpt-realtime-2.1-mini",
"type": "realtime"
},
"value": "ek_..."
}| Request field | Meaning |
|---|---|
session | The settings that a session starts with. session.model names the model. |
model | The model, when session.model is left out. |
expires_after.seconds | The lifetime of the client secret, from 10 to 7200 seconds. The default is 600. A shorter or longer lifetime returns 400 invalid_parameter. |
A request without a model returns 400 missing_required_parameter. A model that doesn't list /v1/realtime in GET /v1/models returns 400 unsupported_endpoint.
value is the client secret, and expires_at is the Unix time when it expires. Until then, anyone who holds the client secret can open sessions that are billed to your account. Use a short lifetime, and create a separate client secret for each visitor.
A client secret works on wss://api.ohmygpt.com/v1/realtime and on POST /v1/realtime/calls. The API key that created it still applies, with its spending limit and its allowed models. If you disable that key, the client secret stops working.
The model is fixed when you create the client secret. A request that names another model returns 400 invalid_parameter. An unknown or expired client secret returns 401:
{
"error": {
"param": null,
"code": "invalid_api_key",
"message": "This client secret is invalid or has expired. Create a new one with `POST /v1/realtime/client_secrets`.",
"type": "authentication_error"
}
}Browsers
Browsers can't set headers on a WebSocket, so pass the client secret as a subprotocol, as OpenAI's browser clients do. The server selects realtime.
// clientSecret is the `value` that your server received
const ws = new WebSocket(
"wss://api.ohmygpt.com/v1/realtime?model=openai/gpt-realtime-2.1-mini",
["realtime", `openai-insecure-api-key.${clientSecret}`],
);Keep API keys on your server. Anyone who can open a page can read a key that the page uses.
WebRTC calls
POST /v1/realtime/calls exchanges a WebRTC SDP offer for an SDP answer. Audio and the oai-events data channel then flow between the client and the model. Send a client secret or an API key in the Authorization header.
// clientSecret is the `value` that your server received
const peer = new RTCPeerConnection();
const speaker = new Audio();
speaker.autoplay = true;
peer.ontrack = (event) => {
speaker.srcObject = event.streams[0];
};
const microphone = await navigator.mediaDevices.getUserMedia({ audio: true });
peer.addTrack(microphone.getTracks()[0], microphone);
const events = peer.createDataChannel("oai-events");
events.onmessage = (event) => console.log(JSON.parse(event.data));
await peer.setLocalDescription(await peer.createOffer());
const form = new FormData();
form.set("sdp", peer.localDescription.sdp);
const response = await fetch("https://api.ohmygpt.com/v1/realtime/calls", {
method: "POST",
headers: { Authorization: "Bearer " + clientSecret },
body: form,
});
await peer.setRemoteDescription({ type: "answer", sdp: await response.text() });The request takes two forms:
Content-Type | Body | Model |
|---|---|---|
multipart/form-data | sdp: the SDP offer. session: the session settings as JSON. | session.model, or the model query parameter |
application/sdp | The SDP offer | The model query parameter |
With a client secret, leave out the model and session. The call then uses the settings that you gave when you created the client secret. With an API key, name the model.
A successful request returns 201 with the SDP answer as application/sdp. These errors are specific to calls:
| Status | Cause |
|---|---|
400 missing_required_parameter | The SDP offer is missing, or an API key was used without a model. |
415 | The Content-Type is neither of the two above. |
429 rate_limit_exceeded | The account already has 10 sessions open. Calls and WebSocket sessions count together. |
When OhMyGPT ends a call, for example at the time limit, it hangs up. A call has no close code or reason.
Transcription sessions
wss://api.ohmygpt.com/v1/realtime?intent=transcription turns live audio into text without generating replies. Use a model that lists this path:
curl -s https://api.ohmygpt.com/v1/models \
| jq -r '.data[] | select([.protocols[].path] | index("/v1/realtime?intent=transcription")) | .id'openai/gpt-4o-mini-transcribe
openai/gpt-4o-transcribe
openai/gpt-live-transcribe
openai/gpt-realtime-whisper
openai/gpt-transcribe
openai/whisper-1The URL has no model parameter. Name the model in a session.update event:
{
"type": "session.update",
"session": {
"type": "transcription",
"audio": {
"input": {
"transcription": { "model": "openai/gpt-4o-mini-transcribe", "language": "en" }
}
}
}
}Send this event within 15 seconds, as one of the first 16 messages and within the first 1 MiB. Otherwise the session closes with code 1008 and Send a session.update with audio.input.transcription.model first.
Transcription sessions take an API key, so open them from your server. A client secret with intent=transcription returns 400 invalid_parameter.
Billing
Each response is billed when it finishes, from the usage in its response.done event, at the model's text, audio, and cached-input prices on its model page. Transcribing input audio (audio.input.transcription) is billed on its own, at the transcription model's price. If you close the connection while a response is being generated, OhMyGPT cancels the response and bills the tokens it used up to that point.
WebRTC calls are billed response by response too. A transcription session is billed at its model's price, per token or per second of audio.
Limits
An account can have 10 sessions open at once. Opening an 11th closes it with code 1008:
websocket: close 1008 (policy violation): Too many concurrent sessions.A session that sends and receives no messages for 10 minutes closes within the next few minutes with code 1000 and Session closed after being idle.
A session lasts at most 60 minutes. It then closes between responses with code 1000 and Session time limit reached.
If the API key is disabled, expires, or reaches its spending limit, its open sessions close between responses with code 1008. The reason is This API key can no longer be used. Check it at https://www.ohmygpt.com/keys.
When OhMyGPT deploys, it closes sessions between responses with code 1012 and Server restarting. Reconnect to continue. Responses that already finished are billed. Open a new session and send your session settings again.
Still stuck? Live chat · [email protected] · Telegram group
Gemini
Call Gemini, and any other chat model, through the Google Gemini generateContent protocol.
Start a Realtime WebRTC call POST
Exchanges a WebRTC SDP offer for an OpenAI Realtime SDP answer. Audio and the `oai-events` data channel then flow between the client and the model; each response is billed as it finishes. Authenticate with an API key or a client secret. Send the offer as `multipart/form-data` with `sdp` and `session` fields, or as `application/sdp` with the model in the `model` query parameter. With a client secret, the model and session settings come from the secret.