AI Gateway - Streamline LLM Integrations

Portkey provides the world's fastest multimodal AI gateway that enables working with multiple LLM providers as well as different modalities very easy. The gateway providers features to improve your app's reliability, cost efficiency and accuracy.

Features

Universal API

Use any of the supported models with a universal API (REST and SDKs)

Cache (Simple & Semantic)

Save costs and decrease latencies by using a cache

Fallbacks

Fallback between providers and models for resilience

Multimodality

Use vision, audio, image generation, and more models

Automatic Retries

Setup automatic retry strategies

Load Balancing

Load balance between various API Keys to counter rate-limits

Canary Testing

Canary test new models in production

Vault

Manage AI provider keys in a secure vault

Request Timeout

Easily handle unresponsive LLM requests

Using the Gateway

The various gateway strategies are implemented using Gateway configs. You can read more about configs below.

pageConfigs

Open Source

We've open sourced our battle-tested AI gateway to the community. You can run it locally with a single command:

npx @portkey-ai/gateway

Contribute here.

You can also self-host the gateway and then connect it to Portkey. Please reach out on [email protected] and we'll help you set this up!