Portkey in November ❄️

We won the NetApp Excellerator Award, launched prompt.new for faster development, added folder organization and AI suggestions for prompt templates, introduced multi-workspace analytics.

Plus, there’s now support for OpenAI’s Realtime API and much more. Let’s dive in!

​
Summary

AreaKey Updates
Platform• See multi-workspace analytics & logs on a single dashboard
• Support for Realtime API across OpenAI and Azure OpenAI
• More granular security & access control settings
• Organize your prompts in folders
Integrations• Route to AWS Sagemaker models through Portkey
• Support for xAI provider and Llama 3.3 & Gemini 2.0 Flash models
• New strictOpenAiCompliance flag on the Gateway
Enterprise• Support for AWS STS with IMDS/IRSA auth
• Support for Azure Entra (formerly Active Directory) to manage Azure auth
• Set budget limits with periodic resets
• Support for any S3-compatible store for logging
Community• Won NetApp’s Best Growth Strategy Award
• Hosted first Practitioners Dinner in Singapore
• Weekly AI Engineering Office Hours

​
Enterprise Spotlight

When API Gateways Don’t Cut It
As AI infrastructure becomes increasingly critical for enterprises, technology leaders are choosing Portkey’s AI Gateway for their AI operations.

Akshay Darbari, Director of Platform Engineering at Premera Blue Cross

When Premera Blue Cross’ Director of Platform Engineering needed an AI Gateway, they chose Portkey. Why? Because traditional API gateways weren’t built for AI-first companies. Are you in the same boat? Schedule an expert consultation here.

​
Platform

​
Prompt Management

  • Type prompt.new in your browser to spin up a new prompt playground! Try it now →
  • Organize your prompt templates with folders and subfolders:
  • Use AI to write and improve your prompts - right inside the playground:
  • Add custom tags/labels like staging, production to any prompt version to track changes, and call them directly:
const promptCompletion = portkey.prompts.completions.create({
    promptID: "pp-article-xx@staging",
    variables: {"":""}
})
  • Each response inside the playground now gives metrics to monitor LLM throughput and latency

​
Analytics

Org-wide Executive Reports

Monitor analytics and logs across all workspaces in your organization through a unified dashboard. This centralized view provides comprehensive insights into cost, performance, and accuracy metrics for your deployed AI applications.

  • Track token usage patterns across requests & responses
  • You can now filter logs and analytics with specific Portkey API keys. This is useful if you are tying a particular key to an internal user and want to see their usage!

​
Enterprise

We’ve strengthened our enterprise authentication capabilities with comprehensive cloud provider integrations.

  • Expanded AWS authentication options, for adding your Bedrock models or Sagemaker deployments:
    • IMDS-based auth (recommended for AWS environments)
    • IRSA-based auth for Kubernetes workloads
    • Role-based auth for non-AWS environments
    • STS integration with assumed roles
  • Also expanded the Azure Integration:
    • Azure Entra (formerly Active Directory)
    • Managed identity support
  • Granular access permissions for API Keys and Virtual Keys across your organization
  • Support for sending Azure deploymentConfig while making Virtual Keys through API. Docs

​
More Customer Love

Felipe & team are building beconfident, and here’s what they had to say about Portkey:

“Now that we’ve seen positive results, we’re going to move all our prompts to Portkey.”

​
Integrations

​
Providers

AWS Sagemaker

Add your Sagemaker deployments to Portkey easily

xAI

Call Grok models through Portkey!

Ollama Tools

Tool calls are now supported on Ollama!

Vertex AI Controlled Generations

The Controlled Generations (read: Structured Outputs) feature on Vertex AI is now supported!

​
Libraries

OpenAI Swarm

Complete observability for Swarm agents

Supabase

Add LLM features to your Supabase apps

Semantic Kernel

Use Portkey in your Microsoft Semantic Kernel apps to easily observe your requests and make them reliable

​
Guardrails

Pangea

Enhanced security with PII detection and content moderation

​
Resources

Essential reading for your AI infrastructure:

​
Community

Our Scaling Story

Building our billion-request architecture

​
Office Hour

One thing we keep hearing from the Portkey community: you want to learn how other teams are solving production challenges and get the most out of the platform. Not through docs or tutorials, but through real conversations with fellow practitioners.

That’s why we’ve started a new series of AI Engineering Hours since last week to bring the Portkey community together to discuss exactly this!

Link to join the next office hour

​
Practitioners’ Dinner

We hosted some of Singapore’s leading Gen AI engineers & leaders for a roundtable conversation - one profound insight emerged: Companies serious about Gen AI have realized it’s as much a platform engineering challenge as it is an AI challenge.

Curious what we mean? Read the meetup note here.

​
Improvements

​
Providers

  • Gemini: Enhanced message and media handling
  • Bedrock: Improved message formatting
  • Vertex AI: Added Zod validation

​
SDK

  • Stream support for assistant threads
  • Enhanced Pydantic compatibility
  • Fixed semantic cache behavior
  • Resolved Python Httpx proxy issues

​
Support

Need Help?

Open an issue on GitHub

Join Us

Get support in our Discord

Special thanks to harupy and Ignacio Gleser for their contributions!