OpenAI

​

Integrate

​

Sample Request

npm install portkey-ai

import Portkey from 'portkey-ai';

const client = new Portkey({
  apiKey: 'PORTKEY_API_KEY',
  provider:'@PROVIDER'
});

async function main() {
  const response = await client.chat.completions.create({
    messages: [{ role: "user", content: "Bob the builder.." }],
    model: "gpt-4o",
  });

  console.log(response.choices[0].message.content);
}

main();

​

Local Setup

npx @portkey-ai/gateway

import Portkey from 'portkey-ai';

const client = new Portkey({
  baseUrl: 'http://localhost:8080/v1',
  apiKey: 'PORTKEY_API_KEY',
  provider:'@PROVIDER'
});

async function main() {
  const response = await client.chat.completions.create({
    messages: [{ role: "user", content: "Bob the builder.." }],
    model: "gpt-4o",
  });

  console.log(response.choices[0].message.content);
}

main();

​

Support for OpenAI Capabilities

​

OpenAI Tool Calling

let tools = [{
    type: "function",
    function: {
        name: "getWeather",
        description: "Get the current weather",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state" },
                unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["location"]
        }
    }
}];

let response = await portkey.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What's the weather like in Delhi - respond in JSON" }
    ],
    tools,
    tool_choice: "auto",
});

console.log(response.choices[0].finish_reason);

​

OpenAI Structured Outputs

​

OpenAI Vision

response = portkey.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
                },
            ],
        }
    ],
    max_tokens=300,
)

print(response)

​

OpenAI Embeddings

response = portkey.embeddings.create(
    input="Your text string goes here",
    model="text-embedding-3-small"
)

print(response.data[0].embedding)

​

OpenAI Prompt Caching

​

OpenAI Image Generations (DALL-E)

• Generate new images from text prompts (DALL·E 3, DALL·E 2)
• Edit existing images with text-guided replacements (DALL·E 2)
• Create variations of existing images (DALL·E 2)

import Portkey from 'portkey-ai';

const client = new Portkey({
  apiKey: 'PORTKEY_API_KEY',
  provider:'@PROVIDER'
});

async function main() {
  const image = await client.images.generate({
    model: "dall-e-3",
    prompt: "Lucy in the sky with diamonds"
  });

  console.log(image.data);
}
main();

​

OpenAI Transcription & Translation (Whisper)

audio_file= open("/path/to/file.mp3", "rb")

# Transcription
transcription = portkey.audio.transcriptions.create(
  model="whisper-1",
  file=audio_file
)
print(transcription.text)

# Translation
translation = portkey.audio.translations.create(
  model="whisper-1",
  file=audio_file
)
print(translation.text)

​

OpenAI Text to Speech

from pathlib import Path

speech_file_path = Path(__file__).parent / "speech.mp3"
response = portkey.audio.speech.create(
  model="tts-1",
  voice="alloy",
  input="Today is a wonderful day to build something people love!"
)

with open(speech_file_path, "wb") as f:
    f.write(response.content)

​

OpenAI Realtime API

• Complete request/response logging for realtime streams
• Cost tracking and budget management for streaming sessions
• Multi-modal conversation monitoring
• Session-based analytics and debugging

• gpt-4o-realtime for full capabilities
• gpt-4o-mini-realtime for lighter applications

​

More Capabilities

​

Portkey Features

Track End-User IDs

Portkey allows you to track user IDs passed with the user parameter in OpenAI requests, enabling you to monitor user-level costs, requests, and more:

Python

Node.js

REST

Copy

Ask AI

response = portkey.chat.completions.create(
  model="gpt-4o",
  messages=[{"role": "user", "content": "Say this is a test"}],
  user="user_123456"
)

When you include the user parameter in your requests, Portkey logs will display the associated user ID, as shown in the image below:In addition to the user parameter, Portkey allows you to send arbitrary custom metadata with your requests. This powerful feature enables you to associate additional context or information with each request, which can be useful for analysis, debugging, or other custom use cases.

Learn More About Metadata

Explore how to use custom metadata to enhance your request tracking and analysis.

Setup Fallbacks & Loadbalancer

Here’s a simplified version of how to use Portkey’s Gateway Configuration:

1

Create a Gateway Configuration

You can create a Gateway configuration using the Portkey Config Dashboard or by writing a JSON configuration in your code. In this example, requests are routed based on the user’s subscription plan (paid or free).

Copy

Ask AI

config = {
  "strategy": {
    "mode": "conditional",
    "conditions": [
      {
        "query": { "metadata.user_plan": { "$eq": "paid" } },
        "then": "gpt4o"
      },
      {
        "query": { "metadata.user_plan": { "$eq": "free" } },
        "then": "gpt-3.5"
      }
    ],
    "default": "base-gpt4"
  },
  "targets": [
    {
      "name": "gpt4o",
      "provider":"@xx"
    },
    {
      "name": "gpt-3.5",
      "provider":"@yy"
    }
  ]
}

2

Process Requests

When a user makes a request, it will pass through Portkey’s AI Gateway. Based on the configuration, the Gateway routes the request according to the user’s metadata.

3

Set Up the Portkey Client

Pass the Gateway configuration to your Portkey client. You can either use the config object or the Config ID from Portkey’s hosted version.

Python

Node.js

Copy

Ask AI

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@PROVIDER",
    config=portkey_config
)

That’s it! Portkey seamlessly allows you to make your AI app more robust using built-in gateway features. Learn more about advanced gateway features:

Load Balancing

Distribute requests across multiple targets based on defined weights.

Fallbacks

Automatically switch to backup targets if the primary target fails.

Conditional Routing

Route requests to different targets based on specified conditions.

Caching

Enable caching of responses to improve performance and reduce costs.

Setup Guardrails

Portkey’s AI gateway enables you to enforce input/output checks on requests by applying custom hooks before and after processing. Protect your user’s/company’s data by using PII guardrails and many more available on Portkey Guardrails:

Copy

Ask AI

{
	"provider:"@openai-xxx",
	"before_request_hooks": [{
		"id": "input-guardrail-id-xx"
	}],
	"after_request_hooks": [{
		"id": "output-guardrail-id-xx"
	}]
}

Learn More About Guardrails

Explore Portkey’s guardrail features to enhance the security and reliability of your AI applications.

Cache Requests

Send Custom Metadata

Send Custom Metadata

Setup Rate Limits

Create & Deploy Prompt Templates

​

Popular Libraries

​

OpenAI with Langchain

​

OpenAI with LangGraph

​

OpenAI with LibreChat

​

OpenAI with CrewAI

​

OpenAI with Llamaindex

​

OpenAI with Vercel

​

More Libraries

​

Cookbooks

​

Appendix

​

OpenAI Projects & Organizations

Managing OpenAI Orgs on Portkey

Organization management is particularly useful if you belong to multiple organizations or are accessing projects through a legacy OpenAI user API key. Specifying the organization and project IDs also helps you maintain better control over your access rules, usage, and costs.In Portkey, you can add your OpenAI Org & Project details by Using Virtual Keys, Using Configs, or While Making a Request.

Using Virtual Keys

When selecting OpenAI from the Virtual Key dropdown menu while creating a virtual key, Portkey displays optional fields for the organization ID and project ID alongside the API key field.

Portkey takes budget management a step further than OpenAI. While OpenAI allows setting budget limits per project, Portkey enables you to set budget limits for each virtual key you create. For more information on budget limits, refer to this documentation

Using Configs

You can also specify the organization and project details in your request config, either at the root level or within a specific target.

Copy

Ask AI

{
	"provider": "openai",
	"api_key": "OPENAI_API_KEY",
	"openai_organization": "org-xxxxxx",
	"openai_project": "proj_xxxxxxxx"
}

While Making a Request

Pass OpenAI organization and project details directly when making a request:

OpenAI Python

OpenAI NodeJS

cURL

Portkey Python

Portkey NodeJS

Copy

Ask AI

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

client = OpenAI(
    api_key="OPENAI_API_KEY",
    organization="org-xxxxxxxxxx",
    project="proj_xxxxxxxxx",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai",
        api_key="PORTKEY_API_KEY"
    )
)

chat_complete = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Say this is a test"}],
)

print(chat_complete.choices[0].message.content)

​

Supported Parameters

​

Supported Models

​

Limitations

​

Limitations for Vision Requests

• Medical images: Vision models are not suitable for interpreting specialized medical images like CT scans and shouldn’t be used for medical advice.
• Non-English: The models may not perform optimally when handling images with text of non-Latin alphabets, such as Japanese or Korean.
• Small text: Enlarge text within the image to improve readability, but avoid cropping important details.
• Rotation: The models may misinterpret rotated / upside-down text or images.
• Visual elements: The models may struggle to understand graphs or text where colors or styles like solid, dashed, or dotted lines vary.
• Spatial reasoning: The models struggle with tasks requiring precise spatial localization, such as identifying chess positions.
• Accuracy: The models may generate incorrect descriptions or captions in certain scenarios.
• Image shape: The models struggle with panoramic and fisheye images.
• Metadata and resizing: The models do not process original file names or metadata, and images are resized before analysis, affecting their original dimensions.
• Counting: May give approximate counts for objects in images.
• CAPTCHAS: For safety reasons, CAPTCHA submissions are blocked by OpenAI.

​

Image Generations Limitations

• DALL·E 3 Restrictions:
  • Only supports image generation (no editing or variations)
  • Limited to one image per request
  • Fixed size options: 1024x1024, 1024x1792, or 1792x1024 pixels
  • Automatic prompt enhancement cannot be disabled
• Image Requirements:
  • Must be PNG format
  • Maximum file size: 4MB
  • Must be square dimensions
  • For edits/variations: input images must meet same requirements
• Content Restrictions:
  • All prompts and images are filtered based on OpenAI’s content policy
  • Violating content will return an error
  • Edited areas must be described in full context, not just the edited portion
• Technical Limitations:
  • Image URLs expire after 1 hour
  • Image editing (inpainting) and variations only available in DALL·E 2
  • Response format limited to URL or Base64 data

​

Speech-to-text Limitations

• File Restrictions:
  • Maximum file size: 25 MB
  • Supported formats: mp3, mp4, mpeg, mpga, m4a, wav, webm
  • No streaming support
• Language Limitations:
  • Translation output available only in English
  • Variable accuracy for non-listed languages
  • Limited control over generated audio compared to other language models
• Technical Constraints:
  • Prompt limited to first 244 tokens
  • Restricted processing for longer audio files
  • No real-time transcription support

​

Text-to-Speech Limitations

• Voice Restrictions:
  • Limited to 6 pre-built voices (alloy, echo, fable, onyx, nova, shimmer)
  • Voices optimized primarily for English
  • No custom voice creation support
  • No direct control over emotional range or tone
• Audio Quality Trade-offs:
  • tts-1: Lower latency but potentially more static
  • tts-1-hd: Higher quality but increased latency
  • Quality differences may vary by listening device
• Usage Requirements:
  • Must disclose AI-generated nature to end users
  • Cannot create custom voice clones
  • Performance varies for non-English languages

​

FAQs

​

General

​

Vision FAQs

​

Embedding FAQs

​

Prompt Caching FAQs

​

Image Generations FAQs

​

Speech-to-text FAQs

​

Text-to-Speech FAQs