Getting Started with AI Gateway

Portkey’s AI Gateway provides a unified interface to 1,600+ LLMs with enterprise features: observability, automatic retries, fallbacks, caching, and cost controls—all through a simple API.

Quick Start

Get started in 3 steps:

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add provider in Model Catalog (e.g., @openai-prod)
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",  # @provider-slug/model-name
    messages=[{"role": "user", "content": "What is a fractal?"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add provider in Model Catalog (e.g., @openai-prod)
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@openai-prod/gpt-4o",  // @provider-slug/model-name
    messages: [{ role: "user", content: "What is a fractal?" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# Use OpenAI SDK with Portkey gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "What is a fractal?"}]
)

print(response.choices[0].message.content)

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

// Use OpenAI SDK with Portkey gateway
const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: PORTKEY_GATEWAY_URL
  })

const response = await client.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{ role: "user", content: "What is a fractal?" }]
})

console.log(response.choices[0].message.content)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
-d '{
    "model": "@openai-prod/gpt-4o",
    "messages": [{"role": "user", "content": "What is a fractal?"}]
}'

Portkey also supports Anthropic’s native /messages endpoint. See Using with Anthropic SDK below.

Add Provider in Model Catalog

Before making requests, add a provider:

Go to Model Catalog → Add Provider
Select your provider (OpenAI, Anthropic, etc.)
Choose existing credentials or enter your API key
Name your provider (e.g., openai-prod)

Your provider slug will be @openai-prod (the name you chose with @ prefix).

Complete Model Catalog Guide →

Set up budgets, rate limits, and manage credentials

Switch Between Providers

Change the model string to use different providers—same code, different models:

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

# OpenAI
response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Anthropic
response = portkey.chat.completions.create(
    model="@anthropic-prod/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=250  # Required for Anthropic
)

# Mistral
response = portkey.chat.completions.create(
    model="@mistral-prod/mistral-large-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)

import Portkey from 'portkey-ai'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

// OpenAI
const openaiResponse = await portkey.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{ role: "user", content: "Hello!" }]
})

// Anthropic
const anthropicResponse = await portkey.chat.completions.create({
    model: "@anthropic-prod/claude-sonnet-4-5-20250929",
    messages: [{ role: "user", content: "Hello!" }],
    max_tokens: 250  // Required for Anthropic
})

// Mistral
const mistralResponse = await portkey.chat.completions.create({
    model: "@mistral-prod/mistral-large-latest",
    messages: [{ role: "user", content: "Hello!" }]
})

Examples

Vision

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"}}
        ]
    }],
    max_tokens=300
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

const response = await portkey.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{
        role: "user",
        content: [
            { type: "text", text: "What's in this image?" },
            { type: "image_url", image_url: { url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" } }
        ]
    }],
    max_tokens: 300
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

client = OpenAI(api_key="PORTKEY_API_KEY", base_url=PORTKEY_GATEWAY_URL)

response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"}}
        ]
    }],
    max_tokens=300
)

print(response.choices[0].message.content)

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

const client = new OpenAI({ apiKey: "PORTKEY_API_KEY", baseURL: PORTKEY_GATEWAY_URL })

const response = await client.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{
        role: "user",
        content: [
            { type: "text", text: "What's in this image?" },
            { type: "image_url", image_url: { url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" } }
        ]
    }],
    max_tokens: 300
})

console.log(response.choices[0].message.content)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-prod/gpt-4o",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"}}
      ]
    }],
    "max_tokens": 300
  }'

Function Calling

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

tools = [{
        "type": "function",
        "function": {
        "name": "get_weather",
        "description": "Get current weather in a location",
          "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
            },
            "required": ["location"]
        }
        }
}]

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Boston?"}],
  tools=tools,
  tool_choice="auto"
)

print(response.choices[0].message.tool_calls)

import Portkey from 'portkey-ai'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

const tools = [{
    type: "function",
    function: {
        name: "get_weather",
        description: "Get current weather in a location",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state, e.g. San Francisco, CA" }
            },
            required: ["location"]
        }
        }
}]

  const response = await portkey.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Boston?" }],
    tools: tools,
    tool_choice: "auto"
})

console.log(response.choices[0].message.tool_calls)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

client = OpenAI(api_key="PORTKEY_API_KEY", base_url=PORTKEY_GATEWAY_URL)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather in a location",
      "parameters": {
        "type": "object",
        "properties": {
                "location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
          },
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Boston?"}],
  tools=tools,
  tool_choice="auto"
)

print(response.choices[0].message.tool_calls)

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

const client = new OpenAI({ apiKey: "PORTKEY_API_KEY", baseURL: PORTKEY_GATEWAY_URL })

const tools = [{
    type: "function",
    function: {
        name: "get_weather",
        description: "Get current weather in a location",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state, e.g. San Francisco, CA" }
            },
            required: ["location"]
        }
    }
}]

const response = await client.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Boston?" }],
    tools: tools,
    tool_choice: "auto"
})

console.log(response.choices[0].message.tool_calls)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-prod/gpt-4o",
    "messages": [{"role": "user", "content": "What is the weather in Boston?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather in a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
          },
          "required": ["location"]
        }
      }
    }],
  "tool_choice": "auto"
  }'

Image Generation

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

image = portkey.images.generate(
    model="@openai-prod/dall-e-3",
    prompt="A serene mountain landscape at sunset",
    size="1024x1024"
)

print(image.data[0].url)

import Portkey from 'portkey-ai'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

const image = await portkey.images.generate({
    model: "@openai-prod/dall-e-3",
    prompt: "A serene mountain landscape at sunset",
    size: "1024x1024"
})

console.log(image.data[0].url)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

client = OpenAI(api_key="PORTKEY_API_KEY", base_url=PORTKEY_GATEWAY_URL)

image = client.images.generate(
    model="@openai-prod/dall-e-3",
    prompt="A serene mountain landscape at sunset",
    size="1024x1024"
)

print(image.data[0].url)

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

const client = new OpenAI({ apiKey: "PORTKEY_API_KEY", baseURL: PORTKEY_GATEWAY_URL })

const image = await client.images.generate({
    model: "@openai-prod/dall-e-3",
    prompt: "A serene mountain landscape at sunset",
    size: "1024x1024"
})

console.log(image.data[0].url)

curl https://api.portkey.ai/v1/images/generations \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-prod/dall-e-3",
    "prompt": "A serene mountain landscape at sunset",
    "size": "1024x1024"
  }'

Embeddings

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

embeddings = portkey.embeddings.create(
    model="@openai-prod/text-embedding-3-small",
    input=["Hello world", "Goodbye world"]
)

print(embeddings.data[0].embedding[:5])  # First 5 dimensions

import Portkey from 'portkey-ai'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

const embeddings = await portkey.embeddings.create({
    model: "@openai-prod/text-embedding-3-small",
    input: ["Hello world", "Goodbye world"]
})

console.log(embeddings.data[0].embedding.slice(0, 5))  // First 5 dimensions

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

client = OpenAI(api_key="PORTKEY_API_KEY", base_url=PORTKEY_GATEWAY_URL)

embeddings = client.embeddings.create(
    model="@openai-prod/text-embedding-3-small",
    input=["Hello world", "Goodbye world"]
)

print(embeddings.data[0].embedding[:5])

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

const client = new OpenAI({ apiKey: "PORTKEY_API_KEY", baseURL: PORTKEY_GATEWAY_URL })

const embeddings = await client.embeddings.create({
    model: "@openai-prod/text-embedding-3-small",
    input: ["Hello world", "Goodbye world"]
})

console.log(embeddings.data[0].embedding.slice(0, 5))

curl https://api.portkey.ai/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-prod/text-embedding-3-small",
    "input": ["Hello world", "Goodbye world"]
  }'

Audio Transcription

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

transcription = portkey.audio.transcriptions.create(
    model="@openai-prod/whisper-1",
    file=open("/path/to/audio.mp3", "rb")
)

print(transcription.text)

import Portkey from 'portkey-ai'
import fs from 'fs'

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" })

const transcription = await portkey.audio.transcriptions.create({
    model: "@openai-prod/whisper-1",
    file: fs.createReadStream("/path/to/audio.mp3")
})

console.log(transcription.text)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

client = OpenAI(api_key="PORTKEY_API_KEY", base_url=PORTKEY_GATEWAY_URL)

transcription = client.audio.transcriptions.create(
    model="@openai-prod/whisper-1",
    file=open("/path/to/audio.mp3", "rb")
)

print(transcription.text)

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'
import fs from 'fs'

const client = new OpenAI({ apiKey: "PORTKEY_API_KEY", baseURL: PORTKEY_GATEWAY_URL })

const transcription = await client.audio.transcriptions.create({
    model: "@openai-prod/whisper-1",
    file: fs.createReadStream("/path/to/audio.mp3")
})

console.log(transcription.text)

curl https://api.portkey.ai/v1/audio/transcriptions \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -F file="@/path/to/audio.mp3" \
  -F model="@openai-prod/whisper-1"

Using with OpenAI SDK

Use your existing OpenAI code with Portkey—just change 2 parameters:

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# Change base_url and api_key
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url=PORTKEY_GATEWAY_URL
)

# Use model with @provider-slug prefix
response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

import OpenAI from 'openai'
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

// Change baseURL and apiKey
const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: PORTKEY_GATEWAY_URL
})

// Use model with @provider-slug prefix
const response = await client.chat.completions.create({
    model: "@openai-prod/gpt-4o",
    messages: [{ role: "user", content: "Hello!" }]
})

Using with Anthropic SDK

Portkey fully supports Anthropic’s native /messages endpoint. Use the Anthropic SDK directly with Portkey:

import anthropic

client = anthropic.Anthropic(
    api_key="dummy", # we will use portkey's provider slug
    default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
    base_url="https://api.portkey.ai"
)

message = client.messages.create(
    model="@anthropic-prod/claude-sonnet-4-5-20250929",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude!"}]
)

print(message.content[0].text)

import Anthropic from '@anthropic-ai/sdk'

const client = new Anthropic({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://api.portkey.ai"
})

const message = await client.messages.create({
    model: "@anthropic-prod/claude-sonnet-4-5-20250929",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello, Claude!" }]
})

console.log(message.content[0].text)

curl https://api.portkey.ai/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@anthropic-prod/claude-sonnet-4-5-20250929",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello, Claude!"}]
  }'

All Portkey features (caching, observability, configs) work with the Anthropic SDK—just add the x-portkey-* headers.

Anthropic Integration Guide →

Learn about prompt caching, extended thinking, and more Anthropic features

Gateway Features

Add production features through configs:

from portkey_ai import Portkey

# Attach a config for retries, caching, fallbacks
portkey = Portkey(
        api_key="PORTKEY_API_KEY",
    config="pc-your-config-id"  # Created in Portkey dashboard
)

Automatic Retries

Retry failed requests with exponential backoff

Fallbacks

Automatically switch to backup providers

Caching

Cache responses to reduce costs and latency

Load Balancing

Distribute requests across multiple providers

Gateway Configs Guide →

Learn how to create and use configs

Supported Integrations

Portkey integrates with the entire AI ecosystem:

LLM Providers

1,600+ models from OpenAI, Anthropic, Google, Mistral, Cohere, and 30+ providers

Agent Frameworks

LangChain, CrewAI, AutoGen, OpenAI Agents, Strands, and more

Libraries

LangChain, LlamaIndex, Vercel AI SDK, and popular frameworks

Guardrails

Aporia, Pillar, Patronus, and content safety providers

Vector Databases

Pinecone, Weaviate, and vector store integrations

MCP Servers

Model Context Protocol servers and tools

Next Steps

Observability

Track costs, latency, and usage

Prompt Library

Manage and version prompts

Guardrails

Add PII detection and content filtering

Model Catalog

Manage providers, budgets, and access

​Quick Start

​Add Provider in Model Catalog

Complete Model Catalog Guide →

​Switch Between Providers

​Examples

​Vision

​Function Calling

​Image Generation

​Embeddings

​Audio Transcription

​Using with OpenAI SDK

​Using with Anthropic SDK

Anthropic Integration Guide →

​Gateway Features

Automatic Retries

Fallbacks

Caching

Load Balancing

Gateway Configs Guide →

​Supported Integrations

LLM Providers

Agent Frameworks

Libraries

Guardrails

Vector Databases

MCP Servers

​Next Steps

Observability

Prompt Library

Guardrails

Model Catalog

Quick Start

Add Provider in Model Catalog

Switch Between Providers

Examples

Vision

Function Calling

Image Generation

Embeddings

Audio Transcription

Using with OpenAI SDK

Using with Anthropic SDK

Gateway Features

Supported Integrations

Next Steps