Summary
Area | Key Updates |
---|---|
Platform | • Prompt Engineering Studio official launch • Support for PDF uploads to Claude • Thinking mode across major providers • University evaluations across Ivy League institutions • 1-click AWS EC2 deployment with CloudFormation |
Gateway | • Multimodal support for Claude (images via URL) • New providers: ncompass and Snowflake Cortex • Enhanced grounding with cached streaming • Improved retry handling and error detection |
Security | • Bring your own encryption key with AWS KMS • SCIM integration for Okta & Azure Entra (AD) • Org-level guardrail and metadata enforcement • Email notifications for usage limits |
Guardrails | • AWS Bedrock Guardrails integration • Mistral Moderations endpoint support • New Guardrail provider: Lasso • New input/output guardrails format |
Documentation | • Admin API documentation • Updated Enterprise Architecture specs • Prompt documentation revamp • Enterprise code visibility in API docs |
Platform
Prompt Engineering Studio
- Version control: Track changes, compare versions, and roll back when needed
- Collaborative workflow: Work together with your team on prompt development
- Variables & templates: Create reusable prompt components and patterns
- Testing framework: Validate performance before production deployment
- Production integration: Seamlessly connect to your applications
- Send image URLs to Claude via Anthropic, Vertex, or Bedrock APIs
- Full support for multimodal conversations and analysis
- Consistent interface across all Claude providers
- Send PDF files directly to Claude requests
- Process long-form documents without manual extraction
- Maintain formatting and structure in analysis
- Support for Anthropic (Bedrock, Vertex), OpenAI, and more
- Full compatibility with streaming responses
- Complete observability of reasoning process
- Consistent interface across all supported models
Enterprise
University Validation We’re proud to announce that Portkey is being evaluated as the official AI Gateway solution by leading academic institutions:- Harvard University
- Princeton University
- University of California, Berkeley
- Cornell University
- New York University
- Lehigh University
- Bowdoin College
- AWS KMS Integration: Bring your own encryption keys for maximum security
- SCIM Support: Automated user provisioning with Okta & Azure Entra (AD)
- Organizational Controls: Enforce guardrails and metadata requirements at the org level
- Usage Limit Notifications: Configure email alerts for rate/budget/usage thresholds
- CloudFormation Template: 1-click deployment of Portkey Gateway on AWS EC2
- Real-Time Model Pricing: Pricing configs now fetched dynamically from control plane
- Internal POD Communication: Secure HTTPS between components
- Enhanced Metrics: Track last byte latency for streaming responses
Gateway & Providers
New ProvidersSnowflake Cortex
Access Snowflake’s AI capabilities through the unified Portkey interface
ncompass
Integration with ncompass AI services
- Enhanced Retry Handling: Better detection of errors in retry process
- Improved Tool Support: Fixed handling of null content for Bedrock tool_calls
- Cached Grounding: Support for cached streaming in grounding requests
- Search Parameters: Support for perplexity.ai search options
- Webhook Enhancement: Return appropriate status codes for streaming webhook failures
Guardrails
We’ve significantly expanded our guardrails capabilities:- AWS Bedrock Guardrails: Native integration with AWS content filtering
- Mistral Moderations: Added support for Mistral’s moderation endpoint
- Lasso Integration: New provider for enhanced content safety
- Input/Output Format: New standardized format for setting guardrails
- Default Headers: Simplified configuration through new API & SDK headers
Documentation
Admin API Introduction
- Admin API Docs: Comprehensive guide to our Control Plane API
- Enterprise Architecture: Updated deployment architecture
- Enterprise Code Visibility: API docs now show code for enterprise deployments
- Prompt Documentation: Complete revamp of our prompt engineering guides
- New Cookbook: Building an LLM as a judge
SDK Updates
- Custom Headers: Send headers with
extra_headers
param in any method - Private Deployment Tracing: Instrument LlamaIndex/LangChain with private deployments
- Support for OpenAI Developer Role: Full compatibility with OpenAI’s new permissions
Analytics
New filtering capabilities in logs & analytics dashboards:- Filter requests by cache status:
- Cache Hit
- Cache Miss
- Cache Disabled
- Cache Semantic Hit
Community
