March

Introducing the Prompt Engineering Studio! 🧪✨

March brings the official launch of our highly anticipated Prompt Engineering Studio – a comprehensive platform for creating, testing, and deploying production-ready prompts with confidence.

We’re also excited to announce that Portkey is now being evaluated as the official AI Gateway solution by several prestigious universities, including Harvard, Princeton, and UC Berkeley.

Additionally, we’ve expanded our multimodal capabilities with Claude image support, added PDF uploads, and introduced thinking mode across major providers. All this with enhanced enterprise security through AWS KMS integration and SCIM for identity management.

Let’s explore all that’s new:

​

Summary

Area	Key Updates
Platform	• Prompt Engineering Studio official launch • Support for PDF uploads to Claude • Thinking mode across major providers • University evaluations across Ivy League institutions • 1-click AWS EC2 deployment with CloudFormation
Gateway	• Multimodal support for Claude (images via URL) • New providers: ncompass and Snowflake Cortex • Enhanced grounding with cached streaming • Improved retry handling and error detection
Security	• Bring your own encryption key with AWS KMS • SCIM integration for Okta & Azure Entra (AD) • Org-level guardrail and metadata enforcement • Email notifications for usage limits
Guardrails	• AWS Bedrock Guardrails integration • Mistral Moderations endpoint support • New Guardrail provider: Lasso • New input/output guardrails format
Documentation	• Admin API documentation • Updated Enterprise Architecture specs • Prompt documentation revamp • Enterprise code visibility in API docs

---

​

Platform

Prompt Engineering Studio

Our flagship release this month is the official launch of the Prompt Engineering Studio, bringing professional-grade prompt development to teams of all sizes:

• Version control: Track changes, compare versions, and roll back when needed
• Collaborative workflow: Work together with your team on prompt development
• Variables & templates: Create reusable prompt components and patterns
• Testing framework: Validate performance before production deployment
• Production integration: Seamlessly connect to your applications

Read about our design journey in our detailed case study.

Claude Multimodal Capabilities

You can now send images to Claude models across various providers:

• Send image URLs to Claude via Anthropic, Vertex, or Bedrock APIs
• Full support for multimodal conversations and analysis
• Consistent interface across all Claude providers

PDF Support for Claude

Enhance your document processing workflows with native PDF support:

• Send PDF files directly to Claude requests
• Process long-form documents without manual extraction
• Maintain formatting and structure in analysis

Thinking Mode Expansion

Access model reasoning across all major providers:

• Support for Anthropic (Bedrock, Vertex), OpenAI, and more
• Full compatibility with streaming responses
• Complete observability of reasoning process
• Consistent interface across all supported models

​

Enterprise

University Validation

We’re proud to announce that Portkey is being evaluated as the official AI Gateway solution by leading academic institutions:

• Harvard University
• Princeton University
• University of California, Berkeley
• Cornell University
• New York University
• Lehigh University
• Bowdoin College

Learn more about the Internet2 NET+ AI service evaluation.

Enhanced Security Controls

• AWS KMS Integration: Bring your own encryption keys for maximum security
• SCIM Support: Automated user provisioning with Okta & Azure Entra (AD)
• Organizational Controls: Enforce guardrails and metadata requirements at the org level
• Usage Limit Notifications: Configure email alerts for rate/budget/usage thresholds

Simplified Deployment

• CloudFormation Template: 1-click deployment of Portkey Gateway on AWS EC2
• Real-Time Model Pricing: Pricing configs now fetched dynamically from control plane
• Internal POD Communication: Secure HTTPS between components
• Enhanced Metrics: Track last byte latency for streaming responses

​

Gateway & Providers

New Providers

Technical Improvements

• Enhanced Retry Handling: Better detection of errors in retry process
• Improved Tool Support: Fixed handling of null content for Bedrock tool_calls
• Cached Grounding: Support for cached streaming in grounding requests
• Search Parameters: Support for perplexity.ai search options
• Webhook Enhancement: Return appropriate status codes for streaming webhook failures

​

Guardrails

We’ve significantly expanded our guardrails capabilities:

• AWS Bedrock Guardrails: Native integration with AWS content filtering
• Mistral Moderations: Added support for Mistral’s moderation endpoint
• Lasso Integration: New provider for enhanced content safety
• Input/Output Format: New standardized format for setting guardrails
• Default Headers: Simplified configuration through new API & SDK headers

​

Documentation

Admin API Introduction

We’ve made significant improvements to our documentation:

• Admin API Docs: Comprehensive guide to our Control Plane API
• Enterprise Architecture: Updated deployment architecture
• Enterprise Code Visibility: API docs now show code for enterprise deployments
• Prompt Documentation: Complete revamp of our prompt engineering guides
• New Cookbook: Building an LLM as a judge

​

SDK Updates

• Custom Headers: Send headers with extra_headers param in any method
• Private Deployment Tracing: Instrument LlamaIndex/LangChain with private deployments
• Support for OpenAI Developer Role: Full compatibility with OpenAI’s new permissions

​

Analytics

New filtering capabilities in logs & analytics dashboards:

• Filter requests by cache status:
  • Cache Hit
  • Cache Miss
  • Cache Disabled
  • Cache Semantic Hit

​

Community

“Describing Portkey as merely useful would be an understatement; it’s a must-have.” - @AManInTech

​

Community Contributors

A special thanks to our community contributors this month:

• urbanonymous
• vineye25
• Ignacio
• Ajay Satish

​

Support