OpenAI - Portkey Docs

The AI Gateway provides a robust and secure gateway to integrate OpenAI’s APIs into your applications, including GPT-4o, o1, DALL·E, Whisper, and more. With the AI Gateway, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

All Models

Full support for GPT-4o, o1, GPT-4, GPT-3.5, and all OpenAI models

All Endpoints

Chat, completions, embeddings, audio, images, and more fully supported

Multi-SDK Support

Use with the OpenAI SDK, REST, or popular frameworks like LangChain

Quick Start

Get OpenAI working in 3 steps:

# 1. Add @openai provider in model catalog
# 2. Use it:

curl https://aigw.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai/gpt-4o",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'

from openai import OpenAI

# 1. Install: pip install openai
# 2. Add @openai provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # AI Gateway API key
    base_url="https://aigw.portkey.ai/v1"
)

response = client.chat.completions.create(
    model="@openai/gpt-4o",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"

// 1. Install: npm install openai
// 2. Add @openai provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // AI Gateway API key
    baseURL: "https://aigw.portkey.ai/v1"
})

const response = await client.chat.completions.create({
    model: "@openai/gpt-4o",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

Tip: You can also send x-portkey-provider: @openai as a header and use just model="gpt-4o" in the request.Legacy support: The virtual_key parameter still works for backwards compatibility.

Add Provider in Model Catalog

Go to Model Catalog → Add Provider
Select OpenAI
Choose existing credentials or create new by entering your OpenAI API key
(Optional) Add your OpenAI Organisation ID and Project ID for better cost tracking
Name your provider (e.g., openai-prod)

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

Basic Usage

Streaming

Stream responses for real-time output in your applications:

Advanced Features

Responses API

OpenAI’s Responses API combines the best of both Chat Completions and Assistants APIs. The AI Gateway fully supports this API.

from openai import OpenAI

client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1"
)

response = client.responses.create(
    model="@openai/gpt-4.1",
    input="Tell me a three sentence bedtime story about a unicorn."
)

print(response)

import OpenAI from 'openai'

const openai = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai/v1"
})

const response = await openai.responses.create({
    model: "@openai/gpt-4.1",
    input: "Tell me a three sentence bedtime story about a unicorn."
})

console.log(response)

The Responses API provides a more flexible foundation for building agentic applications with built-in tools that execute automatically.

Remote MCP support on Responses API

The AI Gateway supports Remote MCP support by OpenAI on its Responses API. Learn More

Streaming with Responses API

response = client.responses.create(
    model="gpt-4.1",
    instructions="You are a helpful assistant.",
    input="Hello!",
    stream=True
)

for event in response:
    print(event)

const response = await openai.responses.create({
    model: "gpt-4.1",
    instructions: "You are a helpful assistant.",
    input: "Hello!",
    stream: true
})

for await (const event of response) {
    console.log(event)
}

Realtime API

The AI Gateway supports OpenAI’s Realtime API with a seamless integration. This allows you to use the AI Gateway’s logging, cost tracking, and guardrail features while using the Realtime API.

Realtime API

Using Vision Models

The AI Gateway’s multimodal Gateway fully supports OpenAI vision models as well. See this guide for more info:

Vision

Vision with the Responses API

The Responses API also processes images alongside text:

response = client.responses.create(
    model="gpt-4.1",
    input=[
        {
            "role": "user",
            "content": [
                { "type": "input_text", "text": "What is in this image?" },
                {
                    "type": "input_image",
                    "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
                }
            ]
        }
    ]
)

print(response)

const response = await openai.responses.create({
    model: "gpt-4.1",
    input: [
        {
            role: "user",
            content: [
                { type: "input_text", text: "What is in this image?" },
                {
                    type: "input_image",
                    image_url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
                }
            ]
        }
    ]
})

console.log(response)

Function Calling

Function calls within your OpenAI SDK operations remain standard. These logs will appear in the AI Gateway, highlighting the utilized functions and their outputs. Additionally, you can define functions within your prompts and call POST /v1/prompts/{promptId}/completions as above.

Function Calling with the Responses API

The Responses API also supports function calling with the same powerful capabilities:

tools = [
    {
        "type": "function",
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location", "unit"]
        }
    }
]

response = client.responses.create(
    model="gpt-4.1",
    tools=tools,
    input="What is the weather like in Boston today?",
    tool_choice="auto"
)

print(response)

const tools = [
    {
        type: "function",
        name: "get_current_weather",
        description: "Get the current weather in a given location",
        parameters: {
            type: "object",
            properties: {
                location: {
                    type: "string",
                    description: "The city and state, e.g. San Francisco, CA"
                },
                unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["location", "unit"]
        }
    }
]

const response = await openai.responses.create({
    model: "gpt-4.1",
    tools: tools,
    input: "What is the weather like in Boston today?",
    tool_choice: "auto"
})

console.log(response)

Fine-Tuning

Please refer to our fine-tuning guides to take advantage of the AI Gateway’s advanced continuous fine-tuning capabilities.

Image Generation

The AI Gateway supports multiple modalities for OpenAI. Make image generation requests through AI Gateway the same way as making completion calls.

// Define the OpenAI client as shown above

const image = await openai.images.generate({
  model:"dall-e-3",
  prompt:"Lucy in the sky with diamonds",
  size:"1024x1024"
})

# Define the OpenAI client as shown above

image = openai.images.generate(
  model="dall-e-3",
  prompt="Lucy in the sky with diamonds",
  size="1024x1024"
)

The AI Gateway’s fast AI gateway captures the information about the request on Strata Cloud Manager. On your logs screen, you’d be able to see this request with the request and response. More information on image generation is available in the API Reference.

Video Generation with Sora

The AI Gateway supports OpenAI’s Sora video generation models through the AI Gateway. Generate videos using the AI Gateway Python SDK:

Pricing for video generation requests will be visible on Strata Cloud Manager, allowing you to track costs alongside your other API usage.

Audio - Transcription, Translation, and Text-to-Speech

The AI Gateway’s multimodal Gateway also supports the audio methods on OpenAI API. Check out the below guides for more info: Check out the below guides for more info:

Text-to-Speech

Speech-to-Text

Integrated Tools with Responses API

Web Search Tool

Web search delivers accurate and clearly-cited answers from the web, using the same tool as search in ChatGPT:

Options for search_context_size:

high: Most comprehensive context, higher cost, slower response
medium: Balanced context, cost, and latency (default)
low: Minimal context, lowest cost, fastest response

Responses include citations for URLs found in search results, with clickable references.

File Search Tool

File search enables quick retrieval from your knowledge base across multiple file types:

This tool requires you to first create a vector store and upload files to it. Supports various file formats including PDFs, DOCXs, TXT, and more. Results include file citations in the response.

Enhanced Reasoning

Control the depth of model reasoning for more comprehensive analysis:

Computer Use Assistant

The AI Gateway also supports the Computer Use Assistant (CUA) tool, which helps agents control computers or virtual machines through screenshots and actions. This feature is available for select developers as a research preview on premium tiers.

Learn More about Computer use tool here

Managing OpenAI Projects & Organisations in the AI Gateway

When integrating OpenAI with the AI Gateway, specify your OpenAI organisation and project IDs along with your API key. This is particularly useful if you belong to multiple organisations or are accessing projects through a legacy user API key. Specifying the organisation and project IDs helps you maintain better control over your access rules, usage, and costs. Add your Org & Project details using:

Adding in Model Catalog (Recommended)
Defining a Gateway Config
Passing Details in a Request

Let’s explore each method in more detail.

Using Model Catalog

When adding OpenAI from the Model Catalog, the AI Gateway automatically displays optional fields for the organisation ID and project ID alongside the API key field. Get your OpenAI API key from here, then add it to the AI Gateway along with your org/project details.

Model Catalog

The AI Gateway takes budget management a step further than OpenAI. While OpenAI allows setting budget limits per project, the AI Gateway enables you to set budget limits for each provider you create. For more information on budget limits, refer to this documentation:

Budget Limits

Using the Gateway Config

You can also specify the organisation and project details in the gateway config, either at the root level or within a specific target.

{
	"provider": "@openai",
	"openai_organization": "org-xxxxxx",
	"openai_project": "proj_xxxxxxxx"
}

While Making a Request

You can also pass your organisation and project details directly when making a request using curl or the OpenAI SDK.

curl https://aigw.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-openai-organization: org-xxxxxxx" \
  -H "x-portkey-openai-project: proj_xxxxxxx" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai/gpt-4o",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

from openai import OpenAI

client = OpenAI(
    api_key="PORTKEY_API_KEY",
    organization="org-xxxxxxxxxx",
    project="proj_xxxxxxxxx",
    base_url="https://aigw.portkey.ai/v1"
)

chat_complete = client.chat.completions.create(
    model="@openai/gpt-4o",
    messages=[{"role": "user", "content": "Say this is a test"}],
)

print(chat_complete.choices[0].message.content)

import OpenAI from "openai"

const openai = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  organization: "org-xxxxxx",
  project: "proj_xxxxxxx",
  baseURL: "https://aigw.portkey.ai/v1"
})

async function main() {
  const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: "user", content: "Say this is a test" }],
    model: "@openai/gpt-4o",
  })

  console.log(chatCompletion.choices)
}

main()

Frequently Asked Questions

General FAQs

How to get the OpenAI API key?

You can sign up to OpenAI here and grab your scoped API key here.

Is it free to use the OpenAI API?

The OpenAI API can be used by signing up to the OpenAI platform. You can find the pricing info here

I am getting rate limited on OpenAI API

You can find your current rate limits imposed by OpenAI here. For more tips, check out this guide.

All Models

All Endpoints

Multi-SDK Support

​Quick Start

​Add Provider in Model Catalog

Complete Setup Guide →

​Basic Usage

​Streaming

​Advanced Features

​Responses API

Remote MCP support on Responses API

​Streaming with Responses API

​Realtime API

Realtime API

​Using Vision Models

​Vision with the Responses API

​Function Calling

​Function Calling with the Responses API

​Fine-Tuning

​Image Generation

​Video Generation with Sora

​Audio - Transcription, Translation, and Text-to-Speech

​Integrated Tools with Responses API

​Web Search Tool

​File Search Tool

​Enhanced Reasoning

​Computer Use Assistant

​Managing OpenAI Projects & Organisations in the AI Gateway

​Using Model Catalog

​Using the Gateway Config

​While Making a Request

​Frequently Asked Questions

​General FAQs

Quick Start

Add Provider in Model Catalog

Basic Usage

Streaming

Advanced Features

Responses API

Streaming with Responses API

Realtime API

Using Vision Models

Vision with the Responses API

Function Calling

Function Calling with the Responses API

Fine-Tuning

Image Generation

Video Generation with Sora

Audio - Transcription, Translation, and Text-to-Speech

Integrated Tools with Responses API

Web Search Tool

File Search Tool

Enhanced Reasoning

Computer Use Assistant

Managing OpenAI Projects & Organisations in the AI Gateway

Using Model Catalog

Using the Gateway Config

While Making a Request

Frequently Asked Questions

General FAQs