December
Ending the year with MCP, intelligence, and enterprise controls! 🛠️
This month we announced our MCP(Model Context Protocol) product - enabling LLMs to leverage 800+ tools through a unified interface. We’ve also added dynamic usage limits on keys, integrated OpenAI’s realtime API, and some new Guardrails. OpenAI’s o1, Llama 3.3 models, Gemini & Perplexity’s grounding features, and the entire HuggingFace model garden on Vertex AI are also available on Portkey now.
For enterprises, we’re introducing comprehensive SSO/SCIM support, enhanced usage controls, and more.
Let’s explore what’s new!
Summary
| Area | Key Updates |
|---|---|
| Platform | • Announced Portkey MCP Client with support for 800+ tools • Set Usage & budget limits for keys • New strict OpenAI compliance mode |
| Integrations | • Support for o1 and Llama 3.3 • Full HuggingFace model garden on Vertex AI • Support for Amazon Nova models • Gemini grounding mode for search-backed responses • Anthropic’s new PDF input capabilities • Microsoft Semantic Kernel integration • Realtime API support |
| Enterprise | • Flexible SSO/SCIM for any OIDC/SAML provider • New workspace management APIs |
| Guardrail | • New guardrail integrations with Promptfoo, and Mistral Moderations • Enhanced regex guardrail capabilities |
Model Context Protocol
Portkey’s Model Context Protocol client enables your AI agents to seamlessly interact with hundreds of tools while maintaining enterprise-grade observability and control.
- Connect to any database or data source
- Build and integrate custom tools
- Execute code safely in controlled environments
- Maintain complete observability and control
All while radically simplifying the complexity of tool calling with MCP.
Join the MCP waitlist →
Platform
Dynamic Usage Limits
We’re introducing comprehensive usage controls for both Virtual Keys and API Keys, giving platform teams precise control over LLM access and resource consumption. This release introduces:
-
Time-based Access Control: Create short-lived keys that automatically expire after a specified duration – perfect for temporary access needs like POCs or time-limited projects
-
Resource Consumption Limits: Set granular limits including:
- Requests per minute (RPM) / Request per hour / Request per day
- Tokens per minute (TPM) / Tokens per hour / Tokens per day
- Budget caps based on cost incurred or tokens consumed, with periodic reset options (weekly/monthly)
Enhanced Provider Features
- Perplexity Integration: Full support for Perplexity API’s advanced features including search domain filtering, related questions generation, and citation capabilities
Browse Docs →
And, there’s more!
- Bulk Prompt Management: Move & Delete multiple prompt templates efficiently
- Enhanced Logging: Automatic language detection in logs view
- Local Gateway Console: Complete request logging with key statistics on the open source Gateway
- Virtual Key API: Programmatically create virtual keys for cloud deployments
Gemini Grounding
Ground LLM responses with real-world data through Google search integration
Anthropic PDF
Native support for PDF processing in Anthropic models, with OpenAI’s image_url field
Realtime API
Complete request and response logging for OpenAI realtime API, including model response, cost, and guardrail violations
Flag for Strict OpenAI Compliance
New flag to toggle provider-specific features while maintaining OpenAI API compatibility
Enterprise
Universal Identity Management
- SSO Integration: Support for all major identity providers through OIDC/SAML standards, enabling seamless enterprise authentication
- Automated User Management: SCIM provisioning for automatic user lifecycle management - from onboarding to role changes and offboarding
- Granular Access Control: Define precise access patterns and manage permissions at both user and workspace levels
- Workspace Management API: Programmatically manage workspaces, user invites, and access controls
Private Deployments
Updated documentation for fully private Portkey installations with enhanced security configurations (Docs)
Integrations
New Providers
HuggingFace on Vertex
Access the complete HuggingFace model garden through Vertex AI
Self-deployed models on Vertex
You can now call your self-deployed models on Vertex AI through Portkey
Amazon Nova
Support for Nova models in prompt playground
Azure AI Inference
Full integration with Azure AI platform
Qdrant
Route your Qdrant vector DB queries through Portkey
Additional Providers
Nebius AI, Inference.net, Voyage AI, Recraft AI
Model & Framework Updates
OpenAI o1
Integrated OpenAI’s latest o1 model across OpenAI & Azure OpenAI
Llama 3.3
Integration with Meta’s latest Llama 3.3 model across multiple providers
Microsoft Semantic Kernel
First-class C# support for Microsoft’s Semantic Kernel framework
Guardrails
Mistral Content Moderation
Content moderation powered by Mistral’s latest model
Promptfoo
Comprehensive evals for jailbreak detection, harmful content, and PII identification
All guardrail responses now include detailed explanations for check results, helping you understand why specific checks passed or failed.
Resources
Essential reading for your AI infrastructure:
- Prompt Injection Attacks: Understanding and preventing security risks
- Real-time vs Batch Evaluation: Choosing the right guardrail strategy
Improvements
- Fixed Cohere streaming on Bedrock
- Improved media support in moderations API
- Enhanced regex guardrail functionality
Support
Was this page helpful?