Overview

Comprehensive documentation for Portkey's AI Gateway, Guardrails, Observability, Prompts, and Governance features.

Portkey Docs

Portkey AI is a comprehensive platform designed to streamline and enhance AI integration for developers and organizations. It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in your Generative AI apps.

What is Portkey?

Integrate Portkey and analyze your first LLM call in 2 minutes!

Make Your First Request

Portkey Features

Automatically create, manage, and execute fine-tuning jobs for Large Language Models (LLMs) across multiple providers.

Autonomous Fine-tuning

Open Source

Comparing Portkey's Open-source version and Dev, Pro, Enterprise plans.

Feature Comparison

Learn how to upgrade to Model Catalog to replace Virtual Keys

Upgrade to Model Catalog

Contact Us

Are you navigating the challenging journey of transitioning LLMs from prototype stages to full-scale production? You're not alone. As this frontier of technology continues to expand, the roadmap isn't always clear. Best practices, guidelines, and efficient methodologies are still on the horizon.

Developer Forum

Since Portkey functions as a gateway - you may encounter Portkey-related, as well as non-Portkey related erros while using our services.

Common Errors & Resolutions

December '23 Migration

Integrations

Submit an Integration

Portkey connects with all major LLM providers and orchestration frameworks.

Google Gemini

Route to your AWS Sagemaker models through Portkey

AWS SageMaker

Ollama

Integrate your privately hosted LLMs with Portkey for unified management, observability, and reliability.

Bring Your Own LLM

Discover how you can build your Gen AI platform on Azure using Portkey

Microsoft Azure

Aporia

Acuvity is model agnostic GenAI security solution. It is built to secure existing and future GenAI models, apps, services, tools, plugins and more.

Acuvity

Integrate Microsoft Azure's powerful content moderation services & PII guardrails with Portkey

Azure Guardrails

Secure your AI applications with AWS Bedrock's guardrail capabilities through Portkey.

AWS Bedrock Guardrails

Lasso Security protects your GenAI apps from data leaks, prompt injections, and other potential risks, keeping your systems safe and secure.

Lasso Security

Mistral moderation service helps detect and filter harmful content across multiple policy dimensions to secure your AI applications.

Mistral

Pangea AI Guard helps analyze and redact text to prevent model manipulation and malicious content.

Pangea

Patronus excels in industry-specific guardrails for RAG workflows.

Patronus AI

Pillar

Prompt Security detects and protects against prompt injection, sensitive data exposure, and other AI security threats.

Prompt Security

Integrate your custom guardrails with Portkey using webhooks

Bring Your Own Guardrails

Transform offline LLMs into online models with real-time internet search capabilities.

Exa Online Search

Milvus

Qdrant

Portkey helps bring your agents to production

Use Portkey with OpenAI Agents SDK to take your AI Agents to production

OpenAI Agents SDK (Python)

OpenAI Agents SDK (TypeScript)

Use Portkey with Autogen to take your AI Agents to production

Autogen

Use Portkey with Control Flow to take your AI Agents to production

Control Flow

Use Portkey with CrewAI to take your AI Agents to production

CrewAI

Langchain Agents

Langroid

Use Portkey with LangGraph to take your AI agent workflows to production

LangGraph

Build production-ready voice AI agents with Portkey's enterprise features

LiveKit

Use Portkey with Llama Agents to take your AI Agents to production

Llama Agents by Llamaindex

The Portkey x Swarm integration brings advanced AI gateway capabilities, full-stack observability, and reliability features to build production-ready AI agents.

OpenAI Swarm

Use Portkey with PydanticAI to take your AI Agents to production

Pydantic AI

Use Portkey with Phidata to take your AI Agents to production

Phidata

Use Portkey with AWS's Strands Agents to take your AI Agents to production

Strands Agents

You can also use Portkey if you are doing custom agent orchestration!

Bring Your own Agents

Add usage tracking, cost controls, and security guardrails to your Anything LLM deployment

Anything LLM

Integrate Portkey with Claude Code for enterprise-grade AI coding assistance with observability and governance

Claude Code

Add enterprise-grade observability, cost tracking, and governance to your Cline AI coding assistant

Cline

Add usage tracking, cost controls, and security guardrails to your Goose AI agent

Goose

Add usage tracking, cost controls, and security guardrails to your Jan deployment

Add usage tracking, cost controls, and security guardrails to your n8n workflows

Add usage tracking, cost controls, and security guardrails to your Codex CLI deployment

OpenAI Codex CLI

Cost tracking, observability, and more for Open WebUI

Open WebUI

Add enterprise-grade observability, cost tracking, and governance to your Roo AI coding assistant

Roo Code

Learn how to integrate Portkey's enterprise features with Zed for enhanced observability, reliability and governance.

AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.

Integrate DSPy with Portkey for production-ready LLM pipelines

DSPy

With Portkey, you can confidently take your Instructor pipelines to production and get complete observability over all of your calls + make them reliable - all with a 2 LOC change!

Instructor

Supercharge Langchain apps with Portkey: Multi-LLM, observability, caching, reliability, and prompt management.

Langchain (Python)

Portkey adds core production capabilities to any Langchain app.

Langchain (JS/TS)

The **Portkey x LlamaIndex** integration brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** to apps built on LlamaIndex.

LlamaIndex (Python)

Portkey brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** + **versioning** to your **Promptfoo** projects. This document provides an overview of how to leverage the strengths of both the platforms to streamline your AI development workflow.

Promptfoo

Integrate Portkey with Vercel AI SDK for production-ready and reliable AI apps

Vercel

Integrate MindsDB with Portkey to build enterprise-grade AI use-cases

MindsDb

ToolJet is a low-code platform that lets you build apps by connecting APIs and data sources, with Portkey integration adding AI features like chat interfaces and automation.

ToolJet

MongoDB

Supabase

Learn how to integrate Portkey's enterprise features with any OpenAI Compliant project for enhanced observability, reliability and governance.

Portkey with Any OpenAI Compatible Project

Microsoft Semantic Kernel

Extend Portkey’s powerful AI Gateway with Arize Phoenix for unified LLM observability, tracing, and analytics across your ML stack.

Arize Phoenix

Logfire is a tool for comprehensive observability of your LLM applications with OpenTelemetry.

Pydantic Logfire

This documentation provides detailed information about the various ways you can access and interact with Portkey - **a robust AI gateway** designed to simplify and enhance your experience with Large Language Models (LLMs) like OpenAI's GPT models. 

Introduction

Supported Providers

Authentication

Header requirements and options for the Portkey API

Headers

Errors

Response Schema

Find the best way to use Portkey in your preferred language.

Supported SDKs

Official Portkey Python SDK to help take your AI apps to production

Python

Official Portkey Node.js SDK – robust, modern, and fully typed integration for JavaScript and TypeScript developers.

Node.js

Integrate Portkey in your `.NET` app easily using the OpenAI library and get advanced monitoring, routing, and enterprise features.

C# (.NET)

Chat

Create a Response

Get a Response

Delete a Response

List Input Items

Execute your saved prompt templates on Portkey


Prompt Completions

Renders a prompt template with its variable values filled in


Prompt Render

Insert a Log

This endpoint allows users to submit feedback for a particular interaction or response.

Prompt Engineering

Whitepapers

Getting Started

Integrations

Use Cases

Overview

How to Run Structure Output Evals on Portkey at Scale

Few-Shot Prompting

Enforcing JSON Schema with Anyscale & Together

Detecting Emotions with GPT-4o

Build an article suggestion app with Supabase pgvector, and Portkey

Setting up resilient Load balancers with failure-mitigating Fallbacks

Run Portkey on Prompts from Langchain Hub

Smart Fallback with Model-Optimized Prompts

How to use OpenAI SDK with Portkey Prompt Templates

Setup OpenAI -> Azure OpenAI Fallback

Fallback from SDXL to Dall-e-3

Comparing Top10 LMSYS Models with Portkey

Build a chatbot using Portkey's Prompt Templates

Track Your LLM Costs Across Users, Teams, Projects, and More Using Portkey Metadata