Portkey + Segmind
Portkey is the Control Panel for AI apps. With it’s popular AI Gateway and Observability Suite, hundreds of teams ship reliable, cost-efficient, and fast apps. With Portkey, you can- Connect to 150+ models through a unified API,
- View 40+ metrics & logs for all requests,
- Enable semantic cache to reduce latency & costs,
- Implement automatic retries & fallbacks for failed requests,
- Add custom tags to requests for better tracking and analysis and more.
Quickstart
Since Portkey is fully compatible with the OpenAI signature, you can connect to the Portkey AI Gateway through OpenAI Client.- Set the
base_url
asPORTKEY_GATEWAY_URL
- Add
default_headers
to consume the headers needed by Portkey using thecreateHeaders
helper method.
With OpenAI Client
With Portkey Client
Optional
Advanced Routing - Fallbacks
The Fallback feature allows you to specify a list of providers/models in a prioritized order. If the primary LLM fails to respond or encounters an error, Portkey will automatically fallback to the next LLM in the list, ensuring your application’s robustness and reliability.
To enable fallbacks, you can modify the config object to include the fallback mode.
Note: You can create and store custom configurations on Portkey.
Monitoring your Requests
Using Portkey you can monitor your Segmind requests and track tokens, cost, latency, and more.
