Open Source
We have open sourced our battle-tested AI Gateway to the community - it connects to 100+ LLMs with a unified interface and a single endpoint, and lets you effortlessly setup fallbacks, load balancing, retries, and more.
This gateway is in production at Portkey processing billions of tokens every day.
Community resource for AI builders to find GPU credits, grants, AI accelerators, or investments - all in a single place. Continuously updated, and sometimes also featuring exclusive deals.
We collaborate with the community to dive deep into how the LLMs & their inference providers are performing at scale, and publish gateway reports. We track latencies, uptime, cost changes, fluctuations across various modalitites like time-of-day, regions, token-lengths, and more.
Report 1: GPT-4 is getting faster
Please reach out on Discord to collaborate on our next report!
Collaborations
Portkey supports various open source projects with additional production capabilities through its custom integrations, and the list is always growing:
Langchain + Portkey - Monitor and trace your Langchain queries
Llamaindex + Portkey - Monitor & trace your requests, and also set up automated fallbacks & load balancing
GPT Prompt Engineer + Portkey - Log all the prompt engineer runs and debug issues easily
Last updated
