Realtime API

OpenAI’s Realtime API while the fastest way to use multi-modal generation, presents its own set of problems around logging, cost tracking and guardrails. AI Gateway provides a solution to these problems with a seamless integration. Its logging is unique in that it captures the entire request and response, including the model’s response, cost, and guardrail violations.

Here’s how to get started:

OpenAI NodeJS
OpenAI Python
cURL

// requires `yarn add ws @types/ws`
import OpenAI from 'openai';
import { OpenAIRealtimeWS } from 'openai/beta/realtime/ws';

const openai = new OpenAI({
  baseURL: "https://aigw.portkey.ai/v1",
  apiKey: "PORTKEY_API_KEY"
});
const rt = new OpenAIRealtimeWS({ model: '@PROVIDER/gpt-4o-realtime-preview-2024-12-17' }, openai);

// access the underlying `ws.WebSocket` instance
rt.socket.on('open', () => {
console.log('Connection opened!');
rt.send({
  type: 'session.update',
  session: {
    modalities: ['text'],
    model: 'gpt-4o-realtime-preview',
  },
});

rt.send({
  type: 'conversation.item.create',
  item: {
    type: 'message',
    role: 'user',
    content: [{ type: 'input_text', text: 'Say a couple paragraphs!' }],
  },
});

rt.send({ type: 'response.create' });
});

rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
// likely want to continue procesing events regardless of any errors
throw err;
});

rt.on('session.created', (event) => {
console.log('session created!', event.session);
console.log();
});

rt.on('response.text.delta', (event) => process.stdout.write(event.delta));
rt.on('response.text.done', () => console.log());

rt.on('response.done', () => rt.close());

rt.socket.on('close', () => console.log('\nConnection closed!'));

import asyncio
from openai import AsyncOpenAI

async def main():
client = AsyncOpenAI(
    base_url="https://aigw.portkey.ai/v1",
    api_key="PORTKEY_API_KEY"
)

async with client.beta.realtime.connect(model="@PROVIDER/gpt-4o-realtime-preview") as connection: #replace with the model you want to use
    await connection.session.update(session={'modalities': ['text']})

    await connection.conversation.item.create(
        item={
            "type": "message",
            "role": "user",
            "content": [{"type": "input_text", "text": "Say hello!"}],
        }
    )
    await connection.response.create()

    async for event in connection:
        if event.type == 'response.text.delta':
            print(event.delta, flush=True, end="")

        elif event.type == 'response.text.done':
            print()

        elif event.type == "response.done":
            break

asyncio.run(main())

# we're using websocat for this example, but you can use any websocket client
websocat "wss://aigw.portkey.ai/v1/realtime?model=gpt-4o-realtime-preview-2024-10-01" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-provider: PROVIDER" \
  -H "x-portkey-OpenAI-Beta: realtime=v1" \
  -H "Authorization: Bearer PORTKEY_API_KEY"
# once connected, you can send your messages as you would with OpenAI's Realtime API

For advanced use cases, you can use configs If you would not like to store your API Keys with the AI Gateway, you can pass your openai key in the Authorization header.

Fire Away!

You can see your logs in realtime with neatly visualized traces and cost tracking.

Introduction

Product

Here’s how to get started:

Fire Away!

Next Steps

​Here’s how to get started:

​Fire Away!

​Next Steps

Here’s how to get started:

Fire Away!

Next Steps