Ending the year with MCP, intelligence, and enterprise controls! 🛠️ This month we announced our MCP(Model Context Protocol) product - enabling LLMs to leverage 800+ tools through a unified interface. We’ve also added dynamic usage limits on keys, integrated OpenAI’s realtime API, and some new Guardrails. OpenAI’s o1, Llama 3.3 models, Gemini & Perplexity’s grounding features, and the entire HuggingFace model garden on Vertex AI are also available on Portkey now. For enterprises, we’re introducing comprehensive SSO/SCIM support, enhanced usage controls, and more. Let’s explore what’s new!

​
Summary

AreaKey Updates
Platform• Announced Portkey MCP Client with support for 800+ tools
• Set Usage & budget limits for keys
• New strict OpenAI compliance mode
Integrations• Support for o1 and Llama 3.3
• Full HuggingFace model garden on Vertex AI
• Support for Amazon Nova models
• Gemini grounding mode for search-backed responses
• Anthropic’s new PDF input capabilities
• Microsoft Semantic Kernel integration
• Realtime API support
Enterprise• Flexible SSO/SCIM for any OIDC/SAML provider

• New workspace management APIs
Guardrail• New guardrail integrations with Promptfoo, and Mistral Moderations
• Enhanced regex guardrail capabilities

​
Model Context Protocol

Portkey’s Model Context Protocol client enables your AI agents to seamlessly interact with hundreds of tools while maintaining enterprise-grade observability and control.
  • Connect to any database or data source
  • Build and integrate custom tools
  • Execute code safely in controlled environments
  • Maintain complete observability and control
All while radically simplifying the complexity of tool calling with MCP.

Join the MCP waitlist →

​
Platform

Dynamic Usage Limits
We’re introducing comprehensive usage controls for both Virtual Keys and API Keys, giving platform teams precise control over LLM access and resource consumption. This release introduces:
  • Time-based Access Control: Create short-lived keys that automatically expire after a specified duration – perfect for temporary access needs like POCs or time-limited projects
  • Resource Consumption Limits: Set granular limits including:
    • Requests per minute (RPM) / Request per hour / Request per day
    • Tokens per minute (TPM) / Tokens per hour / Tokens per day
    • Budget caps based on cost incurred or tokens consumed, with periodic reset options (weekly/monthly)
Enhanced Provider Features
  • Perplexity Integration: Full support for Perplexity API’s advanced features including search domain filtering, related questions generation, and citation capabilities

Browse Docs →

 And, there’s more!
  • Bulk Prompt Management: Move & Delete multiple prompt templates efficiently
  • Enhanced Logging: Automatic language detection in logs view
  • Local Gateway Console: Complete request logging with key statistics on the open source Gateway
  • Virtual Key API: Programmatically create virtual keys for cloud deployments

Gemini Grounding

Ground LLM responses with real-world data through Google search integration

Anthropic PDF

Native support for PDF processing in Anthropic models, with OpenAI’s image_url field

Realtime API

Complete request and response logging for OpenAI realtime API, including model response, cost, and guardrail violations

Flag for Strict OpenAI Compliance

New flag to toggle provider-specific features while maintaining OpenAI API compatibility

​
Enterprise

Universal Identity Management
  • SSO Integration: Support for all major identity providers through OIDC/SAML standards, enabling seamless enterprise authentication
  • Automated User Management: SCIM provisioning for automatic user lifecycle management - from onboarding to role changes and offboarding
  • Granular Access Control: Define precise access patterns and manage permissions at both user and workspace levels
  • Workspace Management API: Programmatically manage workspaces, user invites, and access controls
Private Deployments Updated documentation for fully private Portkey installations with enhanced security configurations (Docs)

​
Integrations

New Providers

HuggingFace on Vertex

Access the complete HuggingFace model garden through Vertex AI

Self-deployed models on Vertex

You can now call your self-deployed models on Vertex AI through Portkey

Amazon Nova

Support for Nova models in prompt playground

Azure AI Inference

Full integration with Azure AI platform

Qdrant

Route your Qdrant vector DB queries through Portkey

Additional Providers

Nebius AI, Inference.net, Voyage AI, Recraft AI
Model & Framework Updates

OpenAI o1

Integrated OpenAI’s latest o1 model across OpenAI & Azure OpenAI

Llama 3.3

Integration with Meta’s latest Llama 3.3 model across multiple providers

Microsoft Semantic Kernel

First-class C# support for Microsoft’s Semantic Kernel framework

​
Guardrails

Mistral Content Moderation

Content moderation powered by Mistral’s latest model

Promptfoo

Comprehensive evals for jailbreak detection, harmful content, and PII identification
All guardrail responses now include detailed explanations for check results, helping you understand why specific checks passed or failed.

​
Resources

Essential reading for your AI infrastructure:

​
Improvements

  • Fixed Cohere streaming on Bedrock
  • Improved media support in moderations API
  • Enhanced regex guardrail functionality

​
Support

Need Help?

Open an issue on GitHub

Join Us

Get support in our Discord