Chat Completions

Available on all Portkey plans.

The Chat Completions API is the most widely adopted format for LLM interaction. Portkey makes it work with every provider — send the same POST /v1/chat/completions request to OpenAI, Anthropic, Gemini, Bedrock, or any of the 3000+ supported models.

Quick Start

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai';

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" });

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{ role: "user", content: "Explain quantum computing in simple terms" }]
});

console.log(response.choices[0].message.content);

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-provider/gpt-4o",
    "messages": [{"role": "user", "content": "Explain quantum computing in simple terms"}]
  }'

Switch model to use any provider — @anthropic-provider/claude-sonnet-4-5-20250514, @google-provider/gemini-2.0-flash, or any of the 3000+ supported models.

Using the OpenAI SDK

The Portkey SDK is a superset of the OpenAI SDK, so all Chat Completions methods work identically. The OpenAI SDK also works directly with Portkey’s base URL:

from openai import OpenAI

client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://api.portkey.ai/v1"
)

response = client.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)

import OpenAI from 'openai';

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://api.portkey.ai/v1"
});

const response = await client.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{ role: "user", content: "Explain quantum computing in simple terms" }]
});

console.log(response.choices[0].message.content);

System Messages

Set a system prompt using the system role in the messages array:

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a pirate. Always respond in pirate speak."},
        {"role": "user", "content": "Say hello."}
    ]
)

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [
        { role: "system", content: "You are a pirate. Always respond in pirate speak." },
        { role: "user", content: "Say hello." }
    ]
});

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-provider/gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a pirate. Always respond in pirate speak."},
      {"role": "user", "content": "Say hello."}
    ]
  }'

Streaming

Stream responses token-by-token with stream: true.

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

stream = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku about AI"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

import Portkey from 'portkey-ai';

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" });

const stream = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{ role: "user", content: "Write a haiku about AI" }],
    stream: true
});

for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
}

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-provider/gpt-4o",
    "messages": [{"role": "user", "content": "Write a haiku about AI"}],
    "stream": true
  }'

Function Calling

Define tools with the tools parameter. Works across all providers that support function calling.

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in San Francisco?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"]
            }
        }
    }]
)

tool_call = response.choices[0].message.tool_calls[0]
print(f"Function: {tool_call.function.name}")
print(f"Arguments: {tool_call.function.arguments}")

import Portkey from 'portkey-ai';

const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY" });

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{ role: "user", content: "What's the weather in San Francisco?" }],
    tools: [{
        type: "function",
        function: {
            name: "get_weather",
            description: "Get current weather for a location",
            parameters: {
                type: "object",
                properties: {
                    location: { type: "string", description: "City name" }
                },
                required: ["location"]
            }
        }
    }]
});

const toolCall = response.choices[0].message.tool_calls[0];
console.log(`Function: ${toolCall.function.name}`);
console.log(`Arguments: ${toolCall.function.arguments}`);

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-provider/gpt-4o",
    "messages": [{"role": "user", "content": "What'\''s the weather in San Francisco?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string", "description": "City name"}
          },
          "required": ["location"]
        }
      }
    }]
  }'

Function Call Results

Pass tool results back to continue the conversation:

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[
        {"role": "user", "content": "What's the weather in Paris?"},
        {"role": "assistant", "tool_calls": [{"id": "call_123", "type": "function", "function": {"name": "get_weather", "arguments": '{"location": "Paris"}'}}]},
        {"role": "tool", "tool_call_id": "call_123", "content": '{"temp": "22°C", "condition": "sunny"}'}
    ]
)

print(response.choices[0].message.content)

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [
        { role: "user", content: "What's the weather in Paris?" },
        { role: "assistant", tool_calls: [{ id: "call_123", type: "function", function: { name: "get_weather", arguments: '{"location": "Paris"}' } }] },
        { role: "tool", tool_call_id: "call_123", content: '{"temp": "22°C", "condition": "sunny"}' }
    ]
});

console.log(response.choices[0].message.content);

Vision

Send images in the content array using the image_url type. Works with all vision-capable models.

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image"},
            {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
        ]
    }]
)

print(response.choices[0].message.content)

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{
        role: "user",
        content: [
            { type: "text", text: "Describe this image" },
            { type: "image_url", image_url: { url: "https://example.com/image.jpg" } }
        ]
    }]
});

console.log(response.choices[0].message.content);

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-provider/gpt-4o",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image"},
        {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
      ]
    }]
  }'

Base64-encoded images are also supported — pass a data URL as the url value:

Python

{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ..."}}

Structured Output

JSON Schema

Force the model to return structured JSON matching a specific schema:

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"}
                },
                "required": ["name", "age"],
                "additionalProperties": False
            }
        }
    }
)

print(response.choices[0].message.content)  # {"name": "John", "age": 30}

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{ role: "user", content: "Extract: John is 30 years old" }],
    response_format: {
        type: "json_schema",
        json_schema: {
            name: "person",
            schema: {
                type: "object",
                properties: {
                    name: { type: "string" },
                    age: { type: "integer" }
                },
                required: ["name", "age"],
                additionalProperties: false
            }
        }
    }
});

console.log(response.choices[0].message.content);

JSON Mode

For free-form JSON output without a strict schema:

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "List 3 programming languages and their main use cases as JSON"}],
    response_format={"type": "json_object"}
)

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [{ role: "user", content: "List 3 programming languages and their main use cases as JSON" }],
    response_format: { type: "json_object" }
});

Multi-turn Conversations

Pass the full conversation history in the messages array:

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "My name is Alice."},
        {"role": "assistant", "content": "Hello Alice! How can I help you?"},
        {"role": "user", "content": "What is my name?"}
    ]
)

print(response.choices[0].message.content)  # "Your name is Alice."

const response = await portkey.chat.completions.create({
    model: "@openai-provider/gpt-4o",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "My name is Alice." },
        { role: "assistant", content: "Hello Alice! How can I help you?" },
        { role: "user", content: "What is my name?" }
    ]
});

console.log(response.choices[0].message.content);

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@openai-provider/gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "My name is Alice."},
      {"role": "assistant", "content": "Hello Alice! How can I help you?"},
      {"role": "user", "content": "What is my name?"}
    ]
  }'

Using with Portkey Features

Chat Completions works with all Portkey gateway features:

Configs — Route, load balance, and set fallbacks
Caching — Cache responses for faster, cheaper calls
Guardrails — Add input/output guardrails
Observability — Full logging and tracing

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    config="pp-config-xxx"  # Config with fallbacks, load balancing, etc.
)

response = portkey.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-config: pp-config-xxx" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

API Reference

Chat Completions — POST /v1/chat/completions

OpenAI Chat API Docs

OpenAI specification

API Reference

Portkey Chat Completions reference

Universal API

All three API formats

Function Calling Guide

Detailed function calling guide

Introduction

Product

Self-Hosting

Support

Quick Start

Using the OpenAI SDK

System Messages

Streaming

Function Calling

Function Call Results

Vision

Structured Output

JSON Schema

JSON Mode

Multi-turn Conversations

Using with Portkey Features

API Reference

OpenAI Chat API Docs

API Reference

Universal API

Function Calling Guide

​Quick Start

​Using the OpenAI SDK

​System Messages

​Streaming

​Function Calling

​Function Call Results

​Vision

​Structured Output

​JSON Schema

​JSON Mode

​Multi-turn Conversations

​Using with Portkey Features

​API Reference

OpenAI Chat API Docs

API Reference

Universal API

Function Calling Guide

Quick Start

Using the OpenAI SDK

System Messages

Streaming

Function Calling

Function Call Results

Vision

Structured Output

JSON Schema

JSON Mode

Multi-turn Conversations

Using with Portkey Features

API Reference