Audio
Audio in, text out, on the OpenAI /v1/audio/transcriptions and /v1/audio/translations shape. Point the OpenAI client at a transcription endpoint and audio.transcriptions.create works unchanged. This is the speech-input surface: transcription in the source language, or translation to English. It also covers the speech-output surface: POST /v1/audio/speech turns text into synthesized audio (see Speech). Audio also rides /v1/chat/completions directly: send audio in as a content part and get a spoken reply back on the same OpenAI shape (see Audio in Chat Completions).
Overview
Two routes convert speech to text:
- Transcriptions,
POST /v1/audio/transcriptions, returns the transcript in the audio's own language. - Translations,
POST /v1/audio/translations, returns the transcript translated to English.
Both routes take a multipart/form-data upload, authenticate with a project API key on the endpoint-scoped path, and render one of five wire formats. The endpoint must be a transcription endpoint (see Endpoint Capability). Requests are routed and billed through the same pipeline as chat completions.
A third route reverses the direction. POST /v1/audio/speech synthesizes audio from text. It is served by a speech endpoint, takes a JSON body, and returns audio bytes or a Server-Sent Events audio stream. See Speech (Text to Speech).
Quick Start
Transcribe a local audio file:
Replace the two highlighted placeholders before running any snippet on this page: the endpoint slug
my-transcribe-endpoint and the API key
ere_myproject_your_api_key. The endpoint must be configured with
task_mode = "transcribe" and backed by a transcription-capable model. Ask your project administrator which transcription endpoint is configured.
curl -X POST https://api.erebine.ai/proj_ABC123/my-transcribe-endpoint/v1/audio/transcriptions \
-H "Authorization: Bearer ere_myproject_your_api_key" \
-F file=@meeting.mp3 \
-F model=whisper-large-v3 \
-F response_format=json
{
"text": "The quarterly review is scheduled for Thursday."
}
Request Format
POST /:project_id/:endpoint_slug/v1/audio/transcriptions
POST /:project_id/:endpoint_slug/v1/audio/translations
Replace :project_id with your project external ID and
:endpoint_slug with the slug of a transcription endpoint configured
in your project. The body is multipart/form-data. The maximum
file size is 25 MB.
Form Fields
| Field | Type | Description |
|---|---|---|
| filerequired | file | The audio file to transcribe. Must be non-empty and at most 25 MB. |
| modelrequired | string | Model identifier. Informational only, the endpoint configuration determines the actual transcription model used. |
| languageoptional | string | Language hint in ISO-639-1 form (for example en, fr). Improves accuracy when the source language is known. Ignored by the translations route, which always targets English. |
| promptoptional | string | Optional text to bias decoding, for example spellings of proper nouns or domain terms. |
| temperatureoptional | number | Decoding temperature between 0 and 1. Lower values are more deterministic. |
| response_formatoptional | string | Output format: json (default), text, srt, verbose_json, or vtt. See Response Formats. |
Response Formats
The response_format field selects one of five shapes.
json (Default)
A single JSON object with the transcript text. Content-Type: application/json.
{ "text": "The quarterly review is scheduled for Thursday." }
text
The raw transcript as text/plain, with no JSON envelope.
The quarterly review is scheduled for Thursday.
verbose_json
A JSON object with the full transcript plus timecoded segments. Includes
task, text, and, when the model reports them,
language, duration (seconds), and a
segments array. Segment timestamps depend on the model: not every
transcription model emits them. A model that does not support
verbose_json returns a 400 (see
Error Handling).
{
"task": "transcribe",
"text": "The quarterly review is scheduled for Thursday.",
"language": "en",
"duration": 3.2,
"segments": [
{
"id": 0,
"start": 0.0,
"end": 3.2,
"text": "The quarterly review is scheduled for Thursday."
}
]
}
srt and vtt
Subtitle files built from the segment timecodes. srt returns SubRip
cues as text/plain; vtt returns a WebVTT document as
text/vtt. Both depend on segment timestamps, so they require a model
that emits them.
1
00:00:00,000 --> 00:00:03,200
The quarterly review is scheduled for Thursday.
Translations
POST /v1/audio/translations takes the same form fields and returns
the transcript translated to English. The verbose_json envelope
reports "task": "translate". The language field is
ignored because the output language is always English.
curl -X POST https://api.erebine.ai/proj_ABC123/my-transcribe-endpoint/v1/audio/translations \
-H "Authorization: Bearer ere_myproject_your_api_key" \
-F file=@interview-fr.mp3 \
-F model=whisper-large-v3
Endpoint Capability
Audio routes are served only by a transcription endpoint: one whose
task_mode is transcribe, backed by a
transcription-capable model. Posting audio to
an endpoint configured for another task mode returns a 400 with code
model_not_transcription.
Create a transcription endpoint from the Endpoints dashboard by selecting a
transcription model and the transcribe task mode, the same flow as
any other endpoint. The endpoint slug in the URL selects which transcription
endpoint serves the request.
Error Handling
Errors use the standard OpenAI error envelope. The table lists the codes specific to the audio routes alongside the generic auth, quota, and capacity errors shared by every endpoint route.
| HTTP Status | Error Code | Description |
|---|---|---|
| Client request errors | ||
| 400 | model_not_transcription |
The target endpoint exists but is not configured for transcription (task_mode is not transcribe). |
| 400 | invalid_request |
The body is not multipart/form-data, the file part is missing or empty, the file exceeds 25 MB, response_format is not one of the five accepted values, temperature is outside 0 to 1, or the model rejected the requested response_format (for example verbose_json on a model that does not support it). |
| 404 | endpoint_not_found |
The :endpoint_slug in the URL does not match any endpoint in the project. |
| Auth and quota | ||
| 401 | authentication_error |
Invalid or missing API key. |
| 402 | insufficient_quota / billing_delinquent |
Project has no remaining credit, or billing is past due. |
| 429 | rate_limit_exceeded |
Too many requests. Check the Retry-After header. |
| Capacity and availability | ||
| 503 | model_provisioning |
The endpoint is starting up and not yet ready to serve traffic. Retry after a short delay. |
| 503 | capacity_exceeded |
No available capacity for the endpoint. Retry after a short delay. |
Client Examples
Python (OpenAI SDK)
Point base_url at the project and endpoint path;
audio.transcriptions.create works unmodified.
from openai import OpenAI
client = OpenAI(
base_url="https://api.erebine.ai/proj_ABC123/my-transcribe-endpoint/v1",
api_key="ere_myproject_your_api_key"
)
with open("meeting.mp3", "rb") as audio:
result = client.audio.transcriptions.create(
model="whisper-large-v3",
file=audio,
response_format="json"
)
print(result.text)
# Translate to English
with open("interview-fr.mp3", "rb") as audio:
english = client.audio.translations.create(
model="whisper-large-v3",
file=audio
)
print(english.text)
curl, verbose_json with segments
curl -X POST \
https://api.erebine.ai/proj_ABC123/my-transcribe-endpoint/v1/audio/transcriptions \
-H "Authorization: Bearer ere_myproject_your_api_key" \
-F file=@meeting.mp3 \
-F model=whisper-large-v3 \
-F response_format=verbose_json \
-F language=en
Realtime Transcription
WS /:project_id/:endpoint_slug/v1/realtime
A WebSocket that streams speech to text as the audio arrives. This is the transcription subset of the OpenAI Realtime API: streaming speech-to-text (ASR), where the client pushes audio buffers up and reads transcript deltas back down the same socket. It is not speech-to-speech and not a realtime voice conversation; there is no spoken model reply on this connection. A stock OpenAI Realtime SDK client driving the transcription flow works against it unchanged.
The endpoint must be a realtime endpoint: one whose
task_mode is realtime, backed by a
transcription-capable model. The socket upgrades on the same
endpoint-scoped path as the other audio routes; replace
:project_id with your project external ID and
:endpoint_slug with the slug of a realtime endpoint
configured in your project.
Authentication
Authenticate the upgrade with a project API key, either way:
- The
Authorization: Bearer <token>header, the same as every other route. - The
Sec-WebSocket-Protocol: bearer.<token>subprotocol, for browser clients that cannot set request headers on a WebSocket.
Client Events
The client sends JSON text frames. Open the session, append audio, then commit the buffer.
| Event | Description |
|---|---|
session.update |
Configures the session. The session object carries "type": "transcription" and names the transcription model. |
input_audio_buffer.append |
Appends a chunk of base64-encoded audio to the input buffer. Send as many as the stream needs. |
input_audio_buffer.commit |
Closes the current audio segment and asks for its transcript. |
{
"type": "session.update",
"session": {
"type": "transcription",
"input_audio_format": "pcm16",
"input_audio_transcription": { "model": "whisper-large-v3" }
}
}
{ "type": "input_audio_buffer.append", "audio": "<base64 pcm16>" }
{ "type": "input_audio_buffer.commit" }
Server Events
The server streams the transcript back as it is produced.
| Event | Description |
|---|---|
conversation.item.input_audio_transcription.delta |
An incremental piece of the transcript in the delta field. Concatenate the deltas in order. |
conversation.item.input_audio_transcription.completed |
The committed segment is fully transcribed. The final text is in the transcript field. |
{ "type": "conversation.item.input_audio_transcription.delta", "delta": "The quarterly" }
{ "type": "conversation.item.input_audio_transcription.completed", "transcript": "The quarterly review is scheduled for Thursday." }
Speech (Text to Speech)
POST /:project_id/:endpoint_slug/v1/audio/speech
Synthesizes audio from text on the OpenAI /v1/audio/speech shape. Point the OpenAI client at a speech endpoint and audio.speech.create works unchanged. The body is JSON. The response is the raw audio bytes, or, with stream_format set to sse, a Server-Sent Events audio stream.
Speech is served only by a speech endpoint: one whose task_mode is speech. Posting to an endpoint configured for another task mode returns a 400 with code model_not_speech. Requests are routed and billed through the same pipeline as chat completions; usage is reported as input tokens derived from the input text and output tokens derived from the produced audio duration.
curl -X POST https://api.erebine.ai/proj_ABC123/my-speech-endpoint/v1/audio/speech \
-H "Authorization: Bearer ere_myproject_your_api_key" \
-H "Content-Type: application/json" \
-d '{"model":"tts-1","input":"The quarterly review is scheduled for Thursday.","voice":"alloy","response_format":"mp3"}' \
--output speech.mp3
Speech Request Format
Replace :project_id with your project external ID and
:endpoint_slug with the slug of a speech endpoint configured in
your project. Send a JSON body with Content-Type: application/json.
Body Fields
| Field | Type | Description |
|---|---|---|
| modelrequired | string | Model identifier. Informational only, the endpoint configuration determines the actual speech model used. |
| inputrequired | string | The text to synthesize. Must be non-empty and at most 4096 characters. |
| voicerequired | string | The voice to speak in. Accepts the OpenAI voice names and the model's native voice names. See Voices. |
| response_formatoptional | string | Audio format: mp3 (default), opus, flac, wav, or pcm. aac is accepted OpenAI vocabulary but is not synthesizable by this endpoint (returns a 400). See Response Modes. |
| speedoptional | number | Playback speed between 0.25 and 4.0. Defaults to 1.0. |
| instructionsoptional | string | Optional style and voice-direction text passed to the model. |
| stream_formatoptional | string | audio for raw audio bytes, or sse for a speech.audio.* Server-Sent Events stream. Streaming requires response_format to be pcm or wav and speed to be 1.0. See Response Modes. |
Voices
The voice field accepts the eleven OpenAI voice names
(alloy, ash, ballad, coral,
echo, fable, nova, onyx,
sage, shimmer, verse), which are mapped
onto the served model's native voices, and the model's native voice names
directly. A voice the model does not recognize returns a 400 with param
voice.
Voices
Which speech model backs an endpoint is a deployment choice, and different
models ship different voices under different names. The portable contract is
the OpenAI voice set: alloy, ash,
ballad, coral, echo,
fable, nova, onyx, sage,
shimmer, and verse. Pass one of these and the
router resolves it to a voice the deployed model actually provides, so the
same request keeps working when the model behind the endpoint changes.
A model's own voice names are accepted too, and take effect directly when the deployed model provides them. They are not portable: a native name that one deployment recognizes is a 400 on another. Prefer the OpenAI names unless you need a specific voice and know which model is deployed. Ask your project administrator which speech endpoint is configured and what it is backed by.
Speech Response Modes
A non-streaming request returns the raw audio bytes with a
Content-Type faithful to the requested
response_format:
| response_format | Content-Type |
|---|---|
mp3 (default) | audio/mpeg |
opus | audio/ogg |
flac | audio/flac |
wav | audio/wav |
pcm | audio/pcm |
stream_format audio
stream_format: "audio" returns the raw audio bytes, the same body
as a non-streaming request.
stream_format sse
stream_format: "sse" returns text/event-stream. The
stream emits a speech.audio.delta event carrying the base64 audio,
followed by a terminal speech.audio.done event carrying the
usage object (input_tokens, output_tokens,
total_tokens). Streaming requires response_format to be
pcm or wav and speed to be 1.0.
event: speech.audio.delta
data: {"type":"speech.audio.delta","audio":"<base64 audio>"}
event: speech.audio.done
data: {"type":"speech.audio.done","usage":{"input_tokens":12,"output_tokens":45,"total_tokens":57}}
Speech Error Handling
Errors use the standard OpenAI error envelope. The table lists the codes specific to the speech route alongside the generic auth, quota, and capacity errors shared by every endpoint route.
| HTTP Status | Error Code | Description |
|---|---|---|
| Client request errors | ||
| 400 | model_not_speech |
The target endpoint exists but is not configured for speech (task_mode is not speech). |
| 400 | invalid_request |
The body is not valid JSON, input is missing/empty or exceeds 4096 characters, voice is missing/empty or not recognized by the model, response_format is not one of the accepted values, response_format is aac (valid OpenAI vocabulary but not synthesizable by this endpoint), speed is outside 0.25 to 4.0, stream_format is not audio or sse, or a streaming request uses a response_format other than pcm/wav or a speed other than 1.0. |
| 404 | endpoint_not_found |
The :endpoint_slug in the URL does not match any endpoint in the project. |
| Auth and quota | ||
| 401 | authentication_error |
Invalid or missing API key. |
| 402 | insufficient_quota / billing_delinquent |
Project has no remaining credit, or billing is past due. |
| 429 | rate_limit_exceeded |
Too many requests. Check the Retry-After header. |
| Capacity and availability | ||
| 503 | model_provisioning |
The endpoint is starting up and not yet ready to serve traffic. Retry after a short delay. |
| 503 | capacity_exceeded |
No available capacity for the endpoint. Retry after a short delay. |
Speech Client Example
Point base_url at the project and endpoint path;
audio.speech.create works unmodified.
from openai import OpenAI
client = OpenAI(
base_url="https://api.erebine.ai/proj_ABC123/my-speech-endpoint/v1",
api_key="ere_myproject_your_api_key"
)
with client.audio.speech.with_streaming_response.create(
model="tts-1",
voice="alloy",
input="The quarterly review is scheduled for Thursday.",
response_format="mp3"
) as response:
response.stream_to_file("speech.mp3")
Audio in Chat Completions
POST /:project_id/:endpoint_slug/v1/chat/completions
Audio travels on the standard chat-completions request, in two independent directions:
- Audio output, a spoken reply. Ask for it with
modalities: ["text", "audio"]and the assistant message carries anaudioobject next to the text. See Audio Output. - Audio input, speech in the prompt. Attach an
input_audiocontent part to a message; it is transcribed before the model sees the turn. See Audio Input.
Each direction needs its own endpoint in the project, separate from the generative endpoint that produces the reply: a spoken reply is synthesized by a speech endpoint, input audio is transcribed by a transcription endpoint. These are the same endpoint kinds documented above for /v1/audio/speech and /v1/audio/transcriptions.
Billing
A spoken reply bills at the speech endpoint's rate, the same as a
/v1/audio/speech call. Transcribed input audio bills transcription
seconds, the same as a /v1/audio/transcriptions call. Both are
metered on top of the text tokens for the turn.
Synthesis length limit
Inline synthesis covers at most the first 4096 characters of the reply, the
same input cap as POST /v1/audio/speech. A longer reply is
truncated for synthesis only: the spoken audio, its transcript,
and the billed audio all reflect the same synthesized text, so they stay
consistent with each other. The text reply in content is never
truncated; only the audio track is bounded.
Audio Output
Two request fields opt a turn into a spoken reply, and they are honored only
together: the audio object is ignored unless
modalities includes audio.
| Field | Type | Description |
|---|---|---|
| modalitiesoptional | array | Output modalities: ["text"] (default) or ["text", "audio"]. Any other modality, such as image or video, returns a 400. |
| audiooptional | object | Voice and format for the spoken reply: { "voice": ..., "format": ... }. Both fields are optional. |
| audio.voiceoptional | string | The synthesis voice. Accepts the same names as /v1/audio/speech: the eleven OpenAI voice names and the model's native voices (see Voices). Defaults to alloy. A voice the speech model cannot resolve returns a 400. |
| audio.formatoptional | string | Container for the returned audio: pcm (default), wav, flac, mp3, or opus. aac is accepted OpenAI vocabulary but is not synthesizable by this endpoint (returns a 400). |
Audio output requires a speech endpoint in the project. If none is provisioned,
the request returns a 400 with code no_audio_endpoint. The voice
and format are validated up front, so an unusable value returns a 400 at request
time rather than a silent text-only reply.
Response
The assistant message on the first choice carries an audio object.
The text reply stays in content.
{
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The quarterly review is scheduled for Thursday.",
"audio": {
"id": "audio_...",
"expires_at": 1737331200,
"data": "<base64 audio>",
"transcript": "The quarterly review is scheduled for Thursday."
}
},
"finish_reason": "stop"
}
]
}
| Field | Type | Description |
|---|---|---|
| id | string | Identifier for the audio segment. |
| expires_at | integer | Unix timestamp after which the server-retained copy of the audio is purged (one hour after the reply). |
| data | string | Base64-encoded audio bytes in the requested format. |
| transcript | string | The text that was synthesized into the audio. |
Usage
When usage is reported, the synthesized audio is counted in
completion_tokens_details.audio_tokens, and that count is included
in completion_tokens and total_tokens, matching
OpenAI's audio models. The text-token count is therefore
completion_tokens minus audio_tokens minus
reasoning_tokens. This token accounting is separate from billing.
import base64
from openai import OpenAI
client = OpenAI(
base_url="https://api.erebine.ai/proj_ABC123/my-endpoint/v1",
api_key="ere_myproject_your_api_key"
)
response = client.chat.completions.create(
model="my-model",
modalities=["text", "audio"],
audio={"voice": "alloy", "format": "wav"},
messages=[
{"role": "user", "content": "Give me a one-line status update."}
]
)
message = response.choices[0].message
print(message.content)
with open("reply.wav", "wb") as out:
out.write(base64.b64decode(message.audio.data))
Streaming Audio Output
With stream: true, the audio arrives on the same
chat.completion.chunk stream as the text, in
choices[0].delta.audio chunks emitted after the last content delta
and before that choice's finish chunk. No new event type is introduced.
The first audio chunk carries the set-once metadata (id,
expires_at, transcript) with no data.
Each following chunk carries one base64 data fragment. Every
fragment is independently base64-decodable, and concatenating the
data fragments in order yields the same bytes as the non-streaming
message.audio.data. Audio chunks carry
finish_reason: null; the finish chunk follows them.
Each chunk below also carries the standard id,
object, created, and model envelope
fields, omitted here for brevity.
data: {"choices":[{"index":0,"delta":{"content":"The quarterly review is Thursday."},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{"audio":{"id":"audio_...","expires_at":1737331200,"transcript":"The quarterly review is Thursday."}},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{"audio":{"data":"<base64 fragment>"}},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
Set stream_options.include_usage to receive the terminal usage
chunk. Its completion_tokens_details.audio_tokens reports the
synthesized audio, folded into completion_tokens and
total_tokens exactly as in the non-streaming body.
Audio Input
A message's content can be an array of parts, and one or more of
those parts can be input_audio. Each audio part is transcribed and
folded into the message text before the model, which is text-only, sees the
turn.
{
"role": "user",
"content": [
{ "type": "text", "text": "Summarize this voice memo." },
{
"type": "input_audio",
"input_audio": { "data": "<base64 audio>", "format": "wav" }
}
]
}
| Field | Type | Description |
|---|---|---|
| input_audio.datarequired | string | Base64-encoded audio bytes. |
| input_audio.formatoptional | string | Declared container, for example wav or mp3. Informational: the transcription model infers the container from the bytes. |
A request may carry at most 8 input_audio parts in total across all
messages; more returns a 400. Transcription runs on the project's transcription
endpoint. If the project has no transcription endpoint, or a transcription
fails, the audio part is dropped and the turn proceeds with whatever text it
has, rather than returning an error.
Chat Audio Error Handling
Errors use the standard OpenAI error envelope. The table lists the codes specific to chat-completions audio, on top of the auth, quota, and capacity errors shared by every endpoint route.
| HTTP Status | Error Code | Description |
|---|---|---|
| Client request errors | ||
| 400 | no_audio_endpoint |
The request asked for the audio modality but the project has no speech endpoint to synthesize it. |
| 400 | unsupported_modality |
modalities names something other than text or audio. |
| 400 | invalid_request |
The audio.voice is not recognized by the speech model, or the audio.format is not an accepted value or is valid OpenAI vocabulary this endpoint cannot synthesize (for example aac). |
| 400 | validation_failed |
The request carries more than 8 input_audio parts. The error details name the messages field with code max_items. |
Chat Dictation
Chat has a microphone button in the composer. It records audio in the browser and transcribes it into the message box. The button appears only when speech-to-text is usable, which requires both:
- A transcription endpoint is available in the project.
- The workspace speech-to-text toggle is on.
Workspace Settings
A workspace owner controls two audio settings in workspace settings:
- Speech-to-text, turns dictation and audio-upload transcription on or off for the workspace.
- Audio retention hours, how long uploaded and recorded audio is kept. Leave it empty to inherit the platform default, or set a number of hours to override it.
When the retention window elapses, the stored audio is removed automatically. The transcript is kept, so a chat that included audio degrades to its transcript rather than losing the content.
Audio Uploads
When speech-to-text is available, audio files attached to a chat are transcribed on upload and the transcript is added to the conversation. Non-audio uploads are unaffected by the audio retention window and remain downloadable.
Free Tier
The free tier includes four endpoint slots, one of which is a transcription endpoint (the others are one generative, one embedding, and one scoring endpoint). Speech is a fifth type; the cap has no room for it. A positive credit balance removes the cap. Creating an endpoint costs nothing; transcription and speech requests bill credits like any other inference. A free subscription includes starter credits to run against. See Usage Tracking & Billing and Billing & Subscriptions for details.
