AI Gateway
Connect to 3,000+ AI models using a single consistent API. Set up load balancers, automated fallbacks, caching, conditional routing, and more, seamlessly.π Universal API
Integrate with multiple AI models through a single API
πΎ Caching
Implement simple and semantic caching for improved performance
π Fallbacks
Set up automated fallbacks for enhanced reliability
π Multimodal Capabilities
Handle various data types with multimodal AI capabilities
π Automatic Retries
Implement automatic retries for improved resilience
π Circuit Breaker
Configure per-strategy circuit protection and failure handling
βοΈ Load Balancing
Distribute workload efficiently across multiple models
π Integrations
Manage access to LLMs
β±οΈ Request Timeouts
Set and manage request timeouts
π€ Canary Tests
Implement canary testing for safe deployments
π¦ Conditional Routing
Route requests based on specific conditions
π³ Budget Limits
Set monthly or custom budget limits on LLM usage, at provider or API key level
β³ Rate Limits
Programmatically set rate limits at API key level and prevent abuse
Observability & Logs
Gain real-time insights, track key metrics, and streamline debugging with our OpenTelemetry-compliant system.π Logs
Access and analyze detailed logs
π΅οΈ Tracing
Implement distributed tracing for request flows
π Analytics
Gain insights through comprehensive analytics
π§ͺ Filters
Apply filters for targeted analysis
π·οΈ Metadata
Manage and utilize metadata effectively
π¬ Feedback
Collect and analyze user feedback
Guardrails
Enforce Real-Time LLM Behavior with 50+ state-of-the-art AI guardrails, so that you can synchronously run Guardrails on your requests and route them with precision.π‘οΈ Deterministic Guardrails
Implement rule-based safety checks
π€ LLM-based Guardrails
Leverage AI for advanced content filtering
π€ Partner Guardrails
Integrate third-party safety solutions
π Bring Your Own
Customize guardrails to your needs
ποΈ Central Guardrails
Deploy org-wide guardrails like PII redaction and company policy adherence across all AI calls
Agents
Natively integrate the AI Gatewayβs routing, guardrails, and observability suite with leading agent frameworks and take them to production.π€ Autogen
π₯ CrewAI
π Langchain Agents
π¦ Llama Agents
Ο PhiData
π ControlFlow
πΈοΈ LangGraph
Governance & Administration
Organise your teams and entities with the access control, encryption, and audit trails that an enterprise AI platform needs.π Org Management
Organise your teams and entities with more access control in the AI Gateway
π SSO Integrations
Seamless integrations with your preferred Single Sign-On providers like Okta and Microsoft
π€ Access Rules
Team and individual level RBAC for AI services, LLMs, logs, and more
ποΈ Audit Logs
System-wide audit trails for all AI services and all the AI Gateway features described above
π KMS Integration
Bring your own encryption keys to the AI Gateway to encrypt data at rest
ποΈ Data Isolation
Ensure the highest level of data privacy and security with isolated storage infrastructure
π Custom Retention Periods
Set custom retention periods for different users to meet your data storage requirements
βοΈ Export to Data Lakes
Easily export all your data to your preferred data lake for long-term storage
π Executive Reporting
Executive-only dashboards to track AI adoption, ROI, and strategic impact across your organisation
More Resources
π¬ Developer Forum
Join our community of developers
π API Reference
Explore our comprehensive API documentation