Links

What is Portkey?

Teams use Portkey to improve the cost, performance, and accuracy of their Gen AI apps.
It takes <2 mins to integrate and with that, it already starts monitoring all of your LLM requests and also makes your app resilient, secure, performant, and more accurate at the same time.
Here's a product walkthrough (3 mins):

Integrate in 3 Lines of Code

OpenAI Python
OpenAI Node
# pip install portkey-ai
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
client = OpenAI(
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(provider="openai", api_key="PORTKEY_API_KEY")
)
chat_complete = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Say this is a test"}],
)
print(chat_complete.choices[0].message.content)
// npm i portkey-ai
import OpenAI from 'openai';
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'
const openai = new OpenAI({
baseURL: PORTKEY_GATEWAY_URL,
defaultHeaders: createHeaders({provider: "openai", apiKey: "PORTKEY_API_KEY"})
});
async function main() {
const chatCompletion = await openai.chat.completions.create({
messages: [{ role: 'user', content: 'Say this is a test' }],
model: 'gpt-3.5-turbo',
});
console.log(chatCompletion.choices);
}
main();

Languages Supported

Language
Supported Library
Javascript
Python
Go
go-openai
Java
Rust
Ruby

AI Providers Supported

AI Provider
Status
OpenAI
fully supported
public
Anthropic
fully supported
public
fully supported
public
Cohere
fully supported
public
Anyscale
fully supported
public
fully supported
public
Google Gemini
fully supported
public
Together AI
fully supported
public
Perplexity
fully supported
public
Mistral
fully supported
public
AWS Bedrock
partially supported
AzureML
partially supported
BYOLLM
partially supported

Frameworks Supported

Framework
Status
Langchain
native
python
native
python