Github Repo

Discuss how Portkey’s AI Gateway can enhance your organization’s AI infrastructure
​
1.8.2
docker pull portkeyai/gateway

​
What’s New

  • Added support for xAI and Sagemaker providers
  • Enhanced proxy support for virtual keys and configs
  • Added citations support for Perplexity through strictOpenAiCompliance flag

​
Improvements

  • Major refactor: Removed deprecated proxy handler code
  • Google Gemini: Improved error message transformation
  • AWS Bedrock: Fixed tool call arguments stringification
​
1.8.1
docker pull portkeyai/gateway

​
What’s New

  • Added support for OpenAI and Azure OpenAI’s Realtime API with complete request logging and cost tracking
  • Expanded Azure authentication options with Azure Entra ID (formerly Azure Active Directory) and Managed Identity support
  • Added new endpoint /v1/reference/models to list all supported models on the Gateway
  • Added new endpoint /v1/reference/providers to list all supported providers on the Gateway
  • Added new Japanese README to the project (community contributed!)
  • New Guardrail: Model Whitelisting to restrict Gateway usage to approved LLMs only

​
Improvements

  • AWS Bedrock: Enhanced message handling by automatically combining consecutive user messages
  • AWS Bedrock: Fixed response formatting by removing redundant newline (\n) characters
  • Vertex AI: Added support for controlled generations via Zod library
  • Azure Openai: Added encoding_format parameter support for embedding requests
​
1.8.0
docker pull portkeyai/gateway

​
Bedrock Converse API integration

  • Bedrock’s /chat/completions have been updated to use Bedrock converse API.
  • This enables features like tool calls, vision, etc. for many bedrock models.
  • This also removes the hassle of maintaining chat templating logic for llama and mistral models.

​
Vertex Image Generation

  • Added support for Vertex Imagen models.

​
Stable Diffusion v2 Models

  • StabilityAI introduced v2 models with a new API signature. Gateway now supports both v1 and v2 models, with internal transformations for different API signatures.
  • Supported for both stability-ai and bedrock providers.
  • New models: Stable Image Ultra, Core, 3.0 and 3.5.

​
Pydantic SDK Integration for Structured Outputs

  • Done for GoogleAI and VertexAI (follows OpenAI)
  • We previously added support for structured outputs through REST API. However, SDKs using Pydantic were not supported due to extra fields in the JSON schema.
  • Added a dereferencing function that converts JSON schemas from the library to Google-compatible schemas.

​
OpenAI and AzureOpenAI Prompt Cache Pricing

  • Added support for handling prompt caching pricing for required models.

​
New Providers

  • Lambda (lambda): Supports chat completions and completions.

​
Fixes & Enhancements:

  • Exclude files, batches, threads, etc. from llm_cost_sum prometheus metric. Apart from the unified routes, all other routes will be excluded from llm_cost_sum metric to avoid unnecessary labels.
  • PerplexityAI: Added the missing [DONE] chunk for stream calls to comply with OpenAI’s spec.
  • VertexAI: Fixed provider name extraction logic for meta models, so users can send it like other partner models (e.g., meta.<model-name>).
  • GoogleAI: Added structured outputs support (similar to Vertex-ai).
  • Updated/Added pricing for new models.

​
Block api.portkey.ai

  • We now block Gateway routes for Enterprise Organisations (Configurable)