May-king it production ready✨
Infra shouldn’t slow you down. In May, we shipped the kind of upgrades that help you move fast into productiion and stay in control—whether you’re scaling agents, securing AI behavior, or managing costs across teams.
From deeper integrations with agent frameworks to support for newer models Portkey keeps evolving as the AI infra layer teams can rely on. We also shipped observability upgrades, expanded our provider network, and added tighter controls for cost, access, and security.
Here’s everything new this month:
Summary
| Area | Key Updates |
|---|---|
| AI agent infrastructure | • PydanticAI integration for modular agent development • OpenAI Agents SDK support with monitoring, guardrails, and cost tracking • Strands Agents integration with observability, retries, and load balancing • Remote MCP server support via Responses API • Arize Phoenix tracing integration for unified agent observability |
| Platform | • Deep integration into Azure AI ecosystem (OpenAI, Foundry, APIM, Marketplace) • Support for Claude Code with rate limits, observability, and access control • AI coding assistant integrations: Cline, Roo Code • Multimodal embedding support via Vertex AI (text, image, video) • Multi-label support for prompt versions • OpenAI Computer Use Tool routing and observability • Full support for GET, PUT, and DELETE HTTP methods• OTel analytics export to your existing observability stack |
| Improvements | • Token cost tracking for gpt-image-1 • Ping messages removed from streamed responses • Resizing metadata columns in logs |
| New models & providers | • Claude 4 now live • PDF support for Claude via Anthropic and Bedrock • OpenAI’s Computer Use Tool supported via Responses API • Grok 3 and Grok 3 Mini on Azure • Gemini 2.5 Thinking Mode in Prompt Playground • Extended thinking for Claude 3.7 and Claude 4 • Mistral supports function calling • WorkersAI supports image generation • Lepton AI, Nscale now integrated • Tool calling enabled for Mistral and OpenRouter • MIME type support for Vertex and Google • PDF support via Anthropic and Bedrock |
| Guardrails | • Prompt Security guardrails for injection detection and sensitive data protection • JWT validator input guardrail • PANW Prisma AIRS plugin for real-time prompt/response risk blocking • Model whitelist guardrail for org/environment/request-level control |
| Documentation | • Guardrail documentation moved under “Integrations” • New solution pages for AWS Bedrock and GovCloud • Cookbook: OpenAI Computer Use tool • Cookbook: Optimizing Prompts using LLama Prompt Ops |
AI agent infrastructure
AI agent frameworks are helping teams prototype faster, but taking agents to production requires real infrastructure. Portkey integrates with leading frameworks to bring interoperability, observability, reliability, and cost management to your agent workflows.
PydanticAI
Portkey now integrates PydanticAI, a Python framework that brings FastAPI-like ergonomics to building AI agents. With Portkey, you can:
- Build modular, testable agents with a clean developer experience.
- Route all agent calls through Portkey for observability and debugging.
- Add retries, fallbacks, guardrails, and cost tracking without extra infra
See how it’s done here
OpenAI Agents SDK
Portkey integrates with the OpenAI Agents SDK to help teams ship production-grade agents with built-in planning, memory, and tool use. You can now:
- Monitor and debug each step of the agent’s reasoning and tool use.
- Automatically track usage and cost for each agent call.
- Apply guardrails to both agent input and output.
- Scale agent-based workflows across environments with versioned control
Explore the integration here
Strands Agents
Strands Agents is a lightweight agent framework built by AWS to simplify agent development.
Portkey now integrates seamlessly with Strands Agents to make them production-ready. With this integration, you get:
- Full observability into agent steps, tool calls, and interactions
- Built-in reliability through fallbacks, retries, and load balancing
- Cost tracking and spend optimization
See how it’s done here
Support for remote MCP servers!
Portkey now supports calling a remote MCP server that is maintained by developers and organizations across the internet that expose these tools to MCP clients via the Responses API Read more about the integrationhere
Tracing Integrations: Arize AI
For teams consolidating observability into Arize, you can now view Portkey’s logs directly into Arize Phoenix to get unified trace views across your LLM workflows.
Platform
Azure AI ecosystem
More than half of Fortune 500 companies use Azure OpenAI. But building GenAI apps in the enterprise is still messy, cost attribution, routing logic, usage tracking, model evaluation… all scattered.
With Portkey’s deep integration into the Azure AI ecosystem (OpenAI, Foundry, APIM, Marketplace), teams can now build, scale, and govern GenAI apps without leaving their existing cloud setup. Our customers are vouching for it!
Claude Code
Bring visibility, governance, and control to Anthropic’s agentic coding assistant with Portkey.
With this integration, you can:
- Avoid system overload by enforcing rate limits
- Monitor usage by tagging and filtering with metadata (e.g., user ID, workspace)
- Debug and trace issues faster with detailed logs for every interaction
- Share controlled, secure access by issuing virtual API keys per user
- Use Claude code in your existing AWS Bedrock or Vertex AI setup, with granular governance and access control
AI coding assistants
Plug Portkey into Cline or Roo Code and enable:
- Access to the latest models from OpenAI, Anthropic, Mistral, and more
- Full observability—log every prompt, tool use, and response with metadata
- Access control with scoped API keys and JWT-based authentication
- Built-in governance and cost tracking per user, project, or team
Multilmodal embeddings
Portkey now supports embedding APIs from Vertex AI for text, image, and video—across multiple languages. This unlocks the ability to:
- Build cross-language search and retrieval
- Power multimodal RAG pipelines
- Track, route, and optimize embedding usage at scale
Read more about the implementation here
Multi-label support for prompts
You can now assign multiple labels to a single prompt version, making it easy to promote a version across environments like staging and production.
OpenAI Computer Use Tool
Build production-grade browser automation with enterprise-level controls using Portkey and:
- Route and monitor Computer Use API calls
- Build a complete Playwright-based browser automation solution
- Add observability, logging, and cost controls with Portkey
Gateway to any API
Portkey now supports GET, PUT, and DELETE HTTP methods in addition to POST, allowing you to route requests to any external or self-hosted provider endpoint. This means you can connect to custom APIs directly through Portkey with full observability for every call.
OTel Integration (Analytics Data) You can now export Portkey analytics to any OpenTelemetry (OTel)-compatible collector, integrating easily into your existing observability stack.
Improvements
- Token cost tracking is now available for gpt-image-1.
- Ping messages are removed from streamed responses.
- Resizing metadata columns in logs
New Models and Providers
- Claude 4 is now live on Portkey.
- PDFs can be sent to Claude via Anthropic and Bedrock.
- OpenAI’s Computer Use Tool works via the Responses API.
- Grok 3 and Grok 3 Mini are available on Azure.
- Gemini 2.5 supports Thinking Mode in Prompt Playground.
- Extended thinking added for Claude 3.7 and Claude 4.
- Mistral now supports function calling.
- Image generation is now available on WorkersAI.
- Lepton AI is now integrated with Portkey.
- Nscale models can be accessed via Portkey.
- Tool calling is live for Mistral and OpenRouter.
- MIME types are now handled for Vertex and Google.
- PDFs are supported via Anthropic and Bedrock routes.
Guardrails
-
Prompt Security guardrails: Integrate with Prompt Security to detect prompt injection and prevent sensitive data exposure in both prompts and responses.
-
JWT validator guardrail: Added as an input guardrail to validate incoming JWT tokens before requests are sent to the LLM.
-
PANW Prisma AIRS Plugin:Portkey now integrates with Palo Alto Networks’ AIRS (AI Runtime Security) to enforce guardrails that block risky prompts or model responses based on real-time security analysis.
-
Model whitelist guardrail:Restrict or deny specific models at the org, environment, or request level using a flexible whitelist/blacklist guardrail.
Documentation and Guides
Optimizing Prompts using LLama Prompt Ops
Need to try out or switch to the latest Llama models? There’s an easier way to do it. Llama Prompt Ops transforms prompts that work well with other LLMs into ones that are optimized specifically for Llama models. This helps you get better performance and more reliable results without having to rewrite everything yourself.
If you work in customer support, we’ve put together a helpful guide that will show you how to build a system that analyzes support messages for urgency and sentiment, and helps categorize them properly.
Check it out here
OpenAI’s Computer Use tool
Build production-grade browser automation with enterprise-level controls. Our latest cookbook shows you how to:
- Route and monitor Computer Use API calls
- Build a complete Playwright-based browser automation solution
- Add observability, logging, and cost controls with Portkey
Other updates
- Guardrail documentation moved under “Integrations”.
- Expanded guides for agent frameworks like CrewAI and LangGraph
Customer love!
Community Contributors
A special thanks to our community contributors this month:
Coming this month!
Provision and manage LLM access across your entire org from a single admin panel. Centralized controls. Granular permissions. Enterprise-grade security. Stay tuned.