---
name: portkey
description: Use when building AI applications that need unified access to 250+
  LLM providers, implementing reliability patterns (fallbacks, retries, load
  balancing), observability (logs, analytics, tracing), guardrails (PII
  detection, content filtering), prompt management, or MCP server integration.
  Essential for production AI systems requiring cost control, latency
  optimization, and security.
metadata:
  mintlify-proj: portkey
  version: "1.0"
---

# Portkey AI Gateway Skill

## Product Summary

Portkey is a unified AI Gateway that routes requests to 250+ LLM providers (OpenAI, Anthropic, Google, Azure, Bedrock, etc.) through a single API. It provides observability (logs, traces, analytics), reliability patterns (fallbacks, retries, load balancing, circuit breakers), guardrails (PII redaction, content filtering), prompt management, and MCP server integration. Agents use Portkey to build resilient, observable, and secure AI applications without vendor lock-in.

**Key files and concepts:**
- **Configs** (`product/ai-gateway/configs`): JSON objects defining routing, fallbacks, caching, retries
- **Integrations** (`product/model-catalog`): Stored provider credentials (OpenAI, Anthropic, etc.)
- **Guardrails** (`product/guardrails`): Input/output validation and security checks
- **Observability** (`product/observability`): Logs, traces, analytics, cost tracking
- **SDKs**: Python (`portkey_ai`), Node.js (`portkey-ai`), or OpenAI SDK with Portkey headers
- **REST API**: `https://api.portkey.ai/v1/` (OpenAI-compatible)
- **Admin API**: `https://api.portkey.ai/admin/` for managing configs, guardrails, integrations

**Primary docs:** https://docs.portkey.ai/docs

---

## When to Use

Reach for this skill when:

- **Routing LLM requests**: Agent needs to send requests to multiple providers or models with fallback logic
- **Building resilient AI apps**: Implementing automatic retries, fallbacks, load balancing, or circuit breakers
- **Cost and latency optimization**: Caching responses, tracking costs per request, or A/B testing models
- **Security and compliance**: Detecting/redacting PII, filtering harmful content, enforcing policies
- **Observability**: Tracking request logs, latency, token usage, errors, or building analytics dashboards
- **Prompt management**: Versioning, testing, and deploying prompts across teams
- **Multi-tenant AI**: Isolating budgets, rate limits, and guardrails per workspace or user
- **MCP integration**: Connecting AI models to external tools via Model Context Protocol servers
- **Provider migration**: Switching between providers without code changes (via configs)

---

## Quick Reference

### Authentication & Setup

| Method | Use Case |
|--------|----------|
| **Portkey SDK** | Recommended for Python/Node.js; handles auth automatically |
| **OpenAI SDK + Portkey headers** | Use existing OpenAI code; add `x-portkey-api-key` header |
| **REST API + headers** | Direct HTTP calls; include `x-portkey-api-key`, `x-portkey-provider` |
| **Config on API key** | Attach default config to API key; applies to all requests with that key |

### Essential Headers

```
x-portkey-api-key: YOUR_PORTKEY_API_KEY
x-portkey-provider: @provider-slug  (e.g., @openai-prod)
x-portkey-config: pc-xxxxx  (config ID or JSON object)
x-portkey-metadata: {"user_id": "123"}  (custom tracking)
```

### Config Structure (JSON)

```json
{
  "strategy": { "mode": "fallback|loadbalance|conditional" },
  "targets": [
    {
      "provider": "@openai-prod",
      "override_params": { "model": "gpt-4o" },
      "default_params": { "temperature": 0.7 },
      "drop_params": ["logprobs"]
    }
  ],
  "retry": { "attempts": 3, "on_status_codes": [429, 500, 502, 503] },
  "cache": { "mode": "simple|semantic", "ttl": 3600 },
  "input_guardrails": ["guardrail-id-xxx"],
  "output_guardrails": ["guardrail-id-yyy"]
}
```

### Common CLI/SDK Patterns

**Python:**
```python
from portkey_ai import Portkey

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY",
    provider="@openai-prod",
    config="pc-xxxxx"  # Optional: config ID
)

response = portkey.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)
```

**Node.js:**
```javascript
import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "YOUR_PORTKEY_API_KEY",
    provider: "@openai-prod",
    config: "pc-xxxxx"
});

const response = await portkey.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello" }]
});
```

**REST API:**
```bash
curl https://api.portkey.ai/v1/chat/completions \
  -H "x-portkey-api-key: YOUR_PORTKEY_API_KEY" \
  -H "x-portkey-provider: @openai-prod" \
  -H "x-portkey-config: pc-xxxxx" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'
```

---

## Decision Guidance

### When to Use X vs Y

| Decision | Use X When | Use Y When |
|----------|-----------|-----------|
| **Fallback vs Load Balance** | One provider fails → try another | Distribute load across healthy providers |
| **Simple vs Semantic Cache** | Exact prompt match | Similar prompts (e.g., "What is AI?" vs "Define AI?") |
| **Async vs Sync Guardrails** | Logging only; no latency impact | Blocking requests on guardrail failure |
| **Config on key vs per-request** | All requests use same routing | Different requests need different strategies |
| **Portkey SDK vs OpenAI SDK** | Full feature access (MCP, advanced routing) | Minimal code changes; OpenAI-compatible |
| **Input vs Output Guardrails** | Validate user prompts before LLM | Validate LLM responses before user |
| **Conditional Routing** | Route based on model, user, metadata | Fixed provider/model routing |

---

## Workflow

### 1. Set Up Provider Credentials (One-Time)

1. Go to **Model Catalog** → **Integrations**
2. Create integration: select provider (OpenAI, Anthropic, etc.), paste API key
3. Create provider: name it (e.g., `@openai-prod`), select integration, set budget/rate limits
4. Note the provider slug for use in configs

### 2. Create a Config (Routing Logic)

1. Go to **Configs** → **Create**
2. Define strategy: fallback, load balance, or conditional routing
3. Add targets: specify providers/models, override params, add guardrails
4. Save and copy config ID (e.g., `pc-xxxxx`)

### 3. Create Guardrails (Optional Security)

1. Go to **Guardrails** → **Create**
2. Add checks: PII detection, regex match, JSON schema, etc.
3. Set actions: async/sync, deny on fail, send feedback
4. Save and copy guardrail ID
5. Add guardrail ID to config's `input_guardrails` or `output_guardrails`

### 4. Make Requests

**Option A: Attach config to API key (default for all requests)**
- Go to **API Keys**, select key, set **Default Config**
- All requests with that key use the config automatically

**Option B: Pass config per-request**
- Python: `portkey.with_options(config="pc-xxxxx").chat.completions.create(...)`
- Node.js: `portkey.chat.completions.create(..., {config: "pc-xxxxx"})`
- REST: Add `x-portkey-config: pc-xxxxx` header

### 5. Monitor & Debug

1. Go to **Logs** → filter by config ID, trace ID, or status
2. Click log entry → view full request/response, guardrail results, latency
3. Go to **Analytics** → view cost, latency, error rate, feedback trends
4. Use **Traces** to follow multi-step requests (e.g., fallback chain)

---

## Common Gotchas

- **Config not applying**: Verify config ID is correct and API key has permission to use it. Request-level config overrides key-level default.
- **Fallback not triggering**: Check `on_status_codes` — fallback only triggers on listed codes (default: all non-2xx). A 400 won't trigger if only [429, 500] are listed.
- **Guardrails blocking all requests**: Ensure guardrail is set to `async: true` or `deny: false` during testing. `deny: true` + `async: false` blocks on any failure.
- **PII redaction not working**: Redaction is irreversible; verify PII categories are enabled in guardrail. Original data is not stored by Portkey.
- **High latency**: Portkey adds 20-40ms. If latency is higher, check: (1) provider response time, (2) guardrail execution time, (3) network location.
- **Costs not tracking**: Ensure metadata is sent with requests (`x-portkey-metadata` header). Costs are calculated from token counts in provider responses.
- **Model not found**: Use format `@provider-slug/model-name` (e.g., `@openai-prod/gpt-4o`). Verify provider slug exists in Model Catalog.
- **Streaming with guardrails**: Output guardrails on streaming responses are informational only; fallback/retry don't trigger. Set `x-portkey-strict-open-ai-compliance: false` to see guardrail results in stream.
- **Rate limit errors**: Use load balancing across multiple API keys or set `on_status_codes: [429]` in fallback to switch providers on rate limit.
- **Prompt versioning confusion**: "Save" creates a new version; "Publish" marks it as production. Always publish before using in production.

---

## Verification Checklist

Before submitting work:

- [ ] **Config created and saved**: Verify config ID in UI or via Admin API
- [ ] **Provider credentials stored**: Integration exists in Model Catalog with valid API key
- [ ] **Guardrails tested**: If using guardrails, test with `async: true` first, then `async: false`
- [ ] **Request headers correct**: `x-portkey-api-key`, `x-portkey-provider`, `x-portkey-config` present
- [ ] **Fallback chain tested**: Simulate primary failure; verify fallback triggers on expected status codes
- [ ] **Logs visible**: Make test request, check Logs page for entry with correct config ID
- [ ] **Cost tracking enabled**: Metadata sent with requests; Analytics shows usage
- [ ] **Latency acceptable**: Check log latency; if >100ms, investigate provider or guardrail performance
- [ ] **Guardrail results visible**: If using guardrails, check log entry for `hook_results` object
- [ ] **Production config locked**: Use `enforce-saved-only-config` to prevent ad-hoc config changes in production

---

## Resources

**Comprehensive navigation:** https://docs.portkey.ai/docs/llms.txt

**Critical pages:**
1. [AI Gateway Overview](https://docs.portkey.ai/docs/product/ai-gateway) — Features, routing strategies
2. [Configs Documentation](https://docs.portkey.ai/docs/product/ai-gateway/configs) — Config schema, examples, passthrough targets
3. [Observability & Logs](https://docs.portkey.ai/docs/product/observability/logs) — Viewing requests, debugging, cost tracking
4. [Guardrails](https://docs.portkey.ai/docs/product/guardrails) — Creating checks, actions, integration with configs
5. [Model Catalog](https://docs.portkey.ai/docs/product/model-catalog) — Managing integrations, providers, budgets, rate limits
6. [Admin API Reference](https://docs.portkey.ai/docs/api-reference/admin-api/introduction) — Programmatic config, guardrail, integration management
7. [SDK Reference](https://docs.portkey.ai/docs/api-reference/sdk/python) — Python, Node.js, C# SDK usage

---

> For additional documentation and navigation, see: https://docs.portkey.ai/docs/llms.txt