Skip to content

Commit e817255

Browse files
stainless-app[bot]stainless-app[bot]
committed
feat(api): realtime API updates
1 parent e566ff3 commit e817255

File tree

27 files changed

+5505
-140
lines changed

27 files changed

+5505
-140
lines changed

‎.stats.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
configured_endpoints: 119
2-
openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8517ffa1004e31ca2523d617629e64be6fe4f13403ddfd9db5b3be002656cbde.yml
3-
openapi_spec_hash: b64dd8c8b23082a7aa2a3e5c5fffd8bd
4-
config_hash: fe0ea26680ac2075a6cd66416aefe7db
1+
configured_endpoints: 118
2+
openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-356b4364203ff36d7724074cd04f6e684253bfcc3c9d969122d730aa7bc51b46.yml
3+
openapi_spec_hash: 4ab8e96f52699bc3d2b0c4432aa92af8
4+
config_hash: b854932c0ea24b400bdd64e4376936bd

‎README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -264,14 +264,14 @@ const { data: stream, request_id } = await openai.chat.completions
264264
.withResponse();
265265
```
266266

267-
## Realtime API Beta
267+
## Realtime API
268268

269269
The Realtime API enables you to build low-latency, multi-modal conversational experiences. It currently supports text and audio as both input and output, as well as [function calling](https://platform.openai.com/docs/guides/function-calling) through a `WebSocket` connection.
270270

271271
```ts
272-
import { OpenAIRealtimeWebSocket } from 'openai/beta/realtime/websocket';
272+
import { OpenAIRealtimeWebSocket } from 'openai/realtime/websocket';
273273

274-
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-4o-realtime-preview-2024-12-17' });
274+
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-realtime' });
275275

276276
rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
277277
```
@@ -401,14 +401,14 @@ while (page.hasNextPage()) {
401401
}
402402
```
403403

404-
## Realtime API Beta
404+
## Realtime API
405405

406406
The Realtime API enables you to build low-latency, multi-modal conversational experiences. It currently supports text and audio as both input and output, as well as [function calling](https://platform.openai.com/docs/guides/function-calling) through a `WebSocket` connection.
407407

408408
```ts
409-
import { OpenAIRealtimeWebSocket } from 'openai/beta/realtime/websocket';
409+
import { OpenAIRealtimeWebSocket } from 'openai/realtime/websocket';
410410

411-
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-4o-realtime-preview-2024-12-17' });
411+
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-realtime' });
412412

413413
rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
414414
```

‎api.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -381,6 +381,7 @@ Types:
381381
- <code><a href="./src/resources/webhooks.ts">FineTuningJobCancelledWebhookEvent</a></code>
382382
- <code><a href="./src/resources/webhooks.ts">FineTuningJobFailedWebhookEvent</a></code>
383383
- <code><a href="./src/resources/webhooks.ts">FineTuningJobSucceededWebhookEvent</a></code>
384+
- <code><a href="./src/resources/webhooks.ts">RealtimeCallIncomingWebhookEvent</a></code>
384385
- <code><a href="./src/resources/webhooks.ts">ResponseCancelledWebhookEvent</a></code>
385386
- <code><a href="./src/resources/webhooks.ts">ResponseCompletedWebhookEvent</a></code>
386387
- <code><a href="./src/resources/webhooks.ts">ResponseFailedWebhookEvent</a></code>
@@ -751,6 +752,7 @@ Types:
751752
- <code><a href="./src/resources/responses/responses.ts">ToolChoiceMcp</a></code>
752753
- <code><a href="./src/resources/responses/responses.ts">ToolChoiceOptions</a></code>
753754
- <code><a href="./src/resources/responses/responses.ts">ToolChoiceTypes</a></code>
755+
- <code><a href="./src/resources/responses/responses.ts">WebSearchPreviewTool</a></code>
754756
- <code><a href="./src/resources/responses/responses.ts">WebSearchTool</a></code>
755757

756758
Methods:
@@ -770,6 +772,110 @@ Methods:
770772

771773
- <code title="get /responses/{response_id}/input_items">client.responses.inputItems.<a href="./src/resources/responses/input-items.ts">list</a>(responseID, { ...params }) -> ResponseItemsPage</code>
772774

775+
# Realtime
776+
777+
Types:
778+
779+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationCreatedEvent</a></code>
780+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItem</a></code>
781+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemAdded</a></code>
782+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemCreateEvent</a></code>
783+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemCreatedEvent</a></code>
784+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemDeleteEvent</a></code>
785+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemDeletedEvent</a></code>
786+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemDone</a></code>
787+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemInputAudioTranscriptionCompletedEvent</a></code>
788+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemInputAudioTranscriptionDeltaEvent</a></code>
789+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemInputAudioTranscriptionFailedEvent</a></code>
790+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemInputAudioTranscriptionSegment</a></code>
791+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemRetrieveEvent</a></code>
792+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemTruncateEvent</a></code>
793+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemTruncatedEvent</a></code>
794+
- <code><a href="./src/resources/realtime/realtime.ts">ConversationItemWithReference</a></code>
795+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferAppendEvent</a></code>
796+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferClearEvent</a></code>
797+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferClearedEvent</a></code>
798+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferCommitEvent</a></code>
799+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferCommittedEvent</a></code>
800+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferSpeechStartedEvent</a></code>
801+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferSpeechStoppedEvent</a></code>
802+
- <code><a href="./src/resources/realtime/realtime.ts">InputAudioBufferTimeoutTriggered</a></code>
803+
- <code><a href="./src/resources/realtime/realtime.ts">LogProbProperties</a></code>
804+
- <code><a href="./src/resources/realtime/realtime.ts">McpListToolsCompleted</a></code>
805+
- <code><a href="./src/resources/realtime/realtime.ts">McpListToolsFailed</a></code>
806+
- <code><a href="./src/resources/realtime/realtime.ts">McpListToolsInProgress</a></code>
807+
- <code><a href="./src/resources/realtime/realtime.ts">OutputAudioBufferClearEvent</a></code>
808+
- <code><a href="./src/resources/realtime/realtime.ts">RateLimitsUpdatedEvent</a></code>
809+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeAudioConfig</a></code>
810+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeClientEvent</a></code>
811+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeClientSecretConfig</a></code>
812+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeConversationItemAssistantMessage</a></code>
813+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeConversationItemFunctionCall</a></code>
814+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeConversationItemFunctionCallOutput</a></code>
815+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeConversationItemSystemMessage</a></code>
816+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeConversationItemUserMessage</a></code>
817+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeError</a></code>
818+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeErrorEvent</a></code>
819+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcpApprovalRequest</a></code>
820+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcpApprovalResponse</a></code>
821+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcpListTools</a></code>
822+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcpProtocolError</a></code>
823+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcpToolCall</a></code>
824+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcpToolExecutionError</a></code>
825+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeMcphttpError</a></code>
826+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeResponse</a></code>
827+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeResponseStatus</a></code>
828+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeResponseUsage</a></code>
829+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeResponseUsageInputTokenDetails</a></code>
830+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeResponseUsageOutputTokenDetails</a></code>
831+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeServerEvent</a></code>
832+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeSession</a></code>
833+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeSessionCreateRequest</a></code>
834+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeToolChoiceConfig</a></code>
835+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeToolsConfig</a></code>
836+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeToolsConfigUnion</a></code>
837+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeTracingConfig</a></code>
838+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeTranscriptionSessionCreateRequest</a></code>
839+
- <code><a href="./src/resources/realtime/realtime.ts">RealtimeTruncation</a></code>
840+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseAudioDeltaEvent</a></code>
841+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseAudioDoneEvent</a></code>
842+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseAudioTranscriptDeltaEvent</a></code>
843+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseAudioTranscriptDoneEvent</a></code>
844+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseCancelEvent</a></code>
845+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseContentPartAddedEvent</a></code>
846+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseContentPartDoneEvent</a></code>
847+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseCreateEvent</a></code>
848+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseCreatedEvent</a></code>
849+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseDoneEvent</a></code>
850+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseFunctionCallArgumentsDeltaEvent</a></code>
851+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseFunctionCallArgumentsDoneEvent</a></code>
852+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseMcpCallArgumentsDelta</a></code>
853+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseMcpCallArgumentsDone</a></code>
854+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseMcpCallCompleted</a></code>
855+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseMcpCallFailed</a></code>
856+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseMcpCallInProgress</a></code>
857+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseOutputItemAddedEvent</a></code>
858+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseOutputItemDoneEvent</a></code>
859+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseTextDeltaEvent</a></code>
860+
- <code><a href="./src/resources/realtime/realtime.ts">ResponseTextDoneEvent</a></code>
861+
- <code><a href="./src/resources/realtime/realtime.ts">SessionCreatedEvent</a></code>
862+
- <code><a href="./src/resources/realtime/realtime.ts">SessionUpdateEvent</a></code>
863+
- <code><a href="./src/resources/realtime/realtime.ts">SessionUpdatedEvent</a></code>
864+
- <code><a href="./src/resources/realtime/realtime.ts">TranscriptionSessionCreated</a></code>
865+
- <code><a href="./src/resources/realtime/realtime.ts">TranscriptionSessionUpdate</a></code>
866+
- <code><a href="./src/resources/realtime/realtime.ts">TranscriptionSessionUpdatedEvent</a></code>
867+
868+
## ClientSecrets
869+
870+
Types:
871+
872+
- <code><a href="./src/resources/realtime/client-secrets.ts">RealtimeSessionCreateResponse</a></code>
873+
- <code><a href="./src/resources/realtime/client-secrets.ts">ClientSecretCreateResponse</a></code>
874+
875+
Methods:
876+
877+
- <code title="post /realtime/client_secrets">client.realtime.clientSecrets.<a href="./src/resources/realtime/client-secrets.ts">create</a>({ ...params }) -> ClientSecretCreateResponse</code>
878+
773879
# Conversations
774880

775881
Types:

‎examples/azure/realtime/websocket.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { OpenAIRealtimeWebSocket } from 'openai/beta/realtime/websocket';
1+
import { OpenAIRealtimeWebSocket } from 'openai/realtime/websocket';
22
import { AzureOpenAI } from 'openai';
33
import { DefaultAzureCredential, getBearerTokenProvider } from '@azure/identity';
44
import 'dotenv/config';
@@ -21,8 +21,9 @@ async function main() {
2121
rt.send({
2222
type: 'session.update',
2323
session: {
24-
modalities: ['text'],
24+
output_modalities: ['text'],
2525
model: 'gpt-4o-realtime-preview',
26+
type: 'realtime',
2627
},
2728
});
2829

@@ -49,8 +50,8 @@ async function main() {
4950
console.log();
5051
});
5152

52-
rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
53-
rt.on('response.text.done', () => console.log());
53+
rt.on('response.output_text.delta', (event) => process.stdout.write(event.delta));
54+
rt.on('response.output_text.done', () => console.log());
5455

5556
rt.on('response.done', () => rt.close());
5657

‎examples/azure/realtime/ws.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { DefaultAzureCredential, getBearerTokenProvider } from '@azure/identity';
2-
import { OpenAIRealtimeWS } from 'openai/beta/realtime/ws';
2+
import { OpenAIRealtimeWS } from 'openai/realtime/ws';
33
import { AzureOpenAI } from 'openai';
44
import 'dotenv/config';
55

@@ -21,8 +21,9 @@ async function main() {
2121
rt.send({
2222
type: 'session.update',
2323
session: {
24-
modalities: ['text'],
24+
output_modalities: ['text'],
2525
model: 'gpt-4o-realtime-preview',
26+
type: 'realtime',
2627
},
2728
});
2829

@@ -49,8 +50,8 @@ async function main() {
4950
console.log();
5051
});
5152

52-
rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
53-
rt.on('response.text.done', () => console.log());
53+
rt.on('response.output_text.delta', (event) => process.stdout.write(event.delta));
54+
rt.on('response.output_text.done', () => console.log());
5455

5556
rt.on('response.done', () => rt.close());
5657

‎examples/realtime/websocket.ts‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,17 @@
1-
import { OpenAIRealtimeWebSocket } from 'openai/beta/realtime/websocket';
1+
import { OpenAIRealtimeWebSocket } from 'openai/realtime/websocket';
22

33
async function main() {
4-
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-4o-realtime-preview-2024-12-17' });
4+
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-realtime' });
55

66
// access the underlying `ws.WebSocket` instance
77
rt.socket.addEventListener('open', () => {
88
console.log('Connection opened!');
99
rt.send({
1010
type: 'session.update',
1111
session: {
12-
modalities: ['text'],
12+
output_modalities: ['text'],
1313
model: 'gpt-4o-realtime-preview',
14+
type: 'realtime',
1415
},
1516
});
1617

@@ -37,8 +38,8 @@ async function main() {
3738
console.log();
3839
});
3940

40-
rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
41-
rt.on('response.text.done', () => console.log());
41+
rt.on('response.output_text.delta', (event) => process.stdout.write(event.delta));
42+
rt.on('response.output_text.done', () => console.log());
4243

4344
rt.on('response.done', () => rt.close());
4445

‎examples/realtime/ws.ts‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,17 @@
1-
import { OpenAIRealtimeWS } from 'openai/beta/realtime/ws';
1+
import { OpenAIRealtimeWS } from 'openai/realtime/ws';
22

33
async function main() {
4-
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
4+
const rt = new OpenAIRealtimeWS({ model: 'gpt-realtime' });
55

66
// access the underlying `ws.WebSocket` instance
77
rt.socket.on('open', () => {
88
console.log('Connection opened!');
99
rt.send({
1010
type: 'session.update',
1111
session: {
12-
modalities: ['text'],
12+
output_modalities: ['text'],
1313
model: 'gpt-4o-realtime-preview',
14+
type: 'realtime',
1415
},
1516
});
1617

@@ -37,8 +38,8 @@ async function main() {
3738
console.log();
3839
});
3940

40-
rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
41-
rt.on('response.text.done', () => console.log());
41+
rt.on('response.output_text.delta', (event) => process.stdout.write(event.delta));
42+
rt.on('response.output_text.done', () => console.log());
4243

4344
rt.on('response.done', () => rt.close());
4445

‎jsr.json.orig‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
".": "./index.ts",
66
"./helpers/zod": "./helpers/zod.ts",
77
"./beta/realtime/websocket": "./beta/realtime/websocket.ts"
8+
"./realtime/websocket": "./realtime/websocket.ts"
89
},
910
"imports": {
1011
"zod": "npm:zod@3"

‎realtime.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
## Realtime API beta
1+
## Realtime API
22

33
The Realtime API enables you to build low-latency, multi-modal conversational experiences. It currently supports text and audio as both input and output, as well as [function calling](https://platform.openai.com/docs/guides/function-calling) through a `WebSocket` connection.
44

@@ -10,9 +10,9 @@ Basic text based example with `ws`:
1010

1111
```ts
1212
// requires `yarn add ws @types/ws`
13-
import { OpenAIRealtimeWS } from 'openai/beta/realtime/ws';
13+
import { OpenAIRealtimeWS } from 'openai/realtime/ws';
1414

15-
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
15+
const rt = new OpenAIRealtimeWS({ model: 'gpt-realtime' });
1616

1717
// access the underlying `ws.WebSocket` instance
1818
rt.socket.on('open', () => {
@@ -59,9 +59,9 @@ rt.socket.on('close', () => console.log('\nConnection closed!'));
5959
To use the web API `WebSocket` implementation, replace `OpenAIRealtimeWS` with `OpenAIRealtimeWebSocket` and adjust any `rt.socket` access:
6060

6161
```ts
62-
import { OpenAIRealtimeWebSocket } from 'openai/beta/realtime/websocket';
62+
import { OpenAIRealtimeWebSocket } from 'openai/realtime/websocket';
6363

64-
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-4o-realtime-preview-2024-12-17' });
64+
const rt = new OpenAIRealtimeWebSocket({ model: 'gpt-realtime' });
6565
// ...
6666
rt.socket.addEventListener('open', () => {
6767
// ...
@@ -77,7 +77,7 @@ When an error is encountered, either on the client side or returned from the ser
7777
It is **highly recommended** that you register an `error` event listener and handle errors appropriately as typically the underlying connection is still usable.
7878

7979
```ts
80-
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
80+
const rt = new OpenAIRealtimeWS({ model: 'gpt-realtime' });
8181
rt.on('error', (err) => {
8282
// in a real world scenario this should be logged somewhere as you
8383
// likely want to continue processing events regardless of any errors

‎scripts/detect-breaking-changes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ TEST_PATHS=(
4444
tests/api-resources/uploads/parts.test.ts
4545
tests/api-resources/responses/responses.test.ts
4646
tests/api-resources/responses/input-items.test.ts
47+
tests/api-resources/realtime/realtime.test.ts
48+
tests/api-resources/realtime/client-secrets.test.ts
4749
tests/api-resources/conversations/conversations.test.ts
4850
tests/api-resources/conversations/items.test.ts
4951
tests/api-resources/evals/evals.test.ts

0 commit comments

Comments
 (0)