Enterprise Releases
Enterprise Gateway
Schedule Call
Discuss how Portkey’s AI Gateway can enhance your organization’s AI infrastructure
1.8.2
What’s New
- Added support for xAI and Sagemaker providers
- Enhanced proxy support for virtual keys and configs
- Added citations support for Perplexity through
strictOpenAiCompliance
flag
Improvements
- Major refactor: Removed deprecated proxy handler code
- Google Gemini: Improved error message transformation
- AWS Bedrock: Fixed tool call arguments stringification
1.8.1
What’s New
- Added support for
OpenAI
andAzure OpenAI
’s Realtime API with complete request logging and cost tracking - Expanded Azure authentication options with Azure Entra ID (formerly Azure Active Directory) and Managed Identity support
- Added new endpoint
/v1/reference/models
to list all supported models on the Gateway - Added new endpoint
/v1/reference/providers
to list all supported providers on the Gateway - Added new Japanese README to the project (community contributed!)
- New Guardrail: Model Whitelisting to restrict Gateway usage to approved LLMs only
Improvements
- AWS Bedrock: Enhanced message handling by automatically combining consecutive user messages
- AWS Bedrock: Fixed response formatting by removing redundant newline (
\n
) characters - Vertex AI: Added support for controlled generations via Zod library
- Azure Openai: Added
encoding_format
parameter support for embedding requests
1.8.0
Bedrock Converse API integration
- Bedrock’s /chat/completions have been updated to use Bedrock converse API.
- This enables features like tool calls, vision, etc. for many bedrock models.
- This also removes the hassle of maintaining chat templating logic for llama and mistral models.
Vertex Image Generation
- Added support for Vertex Imagen models.
Stable Diffusion v2 Models
- StabilityAI introduced v2 models with a new API signature. Gateway now supports both v1 and v2 models, with internal transformations for different API signatures.
- Supported for both stability-ai and bedrock providers.
- New models: Stable Image Ultra, Core, 3.0 and 3.5.
Pydantic SDK Integration for Structured Outputs
- Done for GoogleAI and VertexAI (follows OpenAI)
- We previously added support for structured outputs through REST API. However, SDKs using Pydantic were not supported due to extra fields in the JSON schema.
- Added a dereferencing function that converts JSON schemas from the library to Google-compatible schemas.
OpenAI and AzureOpenAI Prompt Cache Pricing
- Added support for handling prompt caching pricing for required models.
New Providers
- Lambda (
lambda
): Supports chat completions and completions.
Fixes & Enhancements:
- Exclude files, batches, threads, etc. from llm_cost_sum prometheus metric. Apart from the unified routes, all other routes will be excluded from llm_cost_sum metric to avoid unnecessary labels.
- PerplexityAI: Added the missing [DONE] chunk for stream calls to comply with OpenAI’s spec.
- VertexAI: Fixed provider name extraction logic for meta models, so users can send it like other partner models (e.g., meta.
<model-name>
). - GoogleAI: Added structured outputs support (similar to Vertex-ai).
- Updated/Added pricing for new models.
Block api.portkey.ai
- We now block Gateway routes for Enterprise Organisations (Configurable)
Was this page helpful?