Portkey & Patronus - Bringing Responsible LLMs in Production
Patronus AI's suite of evaluators are now available on the Portkey Gateway.
Patronus AI is the world's leading evaluations & guardrails platform for AI apps. They are behind tools like EnterprisePII (dataset for detecting business-sensitive info), FinanceBench (LLM benchmark for financial questions), Lynx (open LLM to detect real-time hallucinations), and more.
Portkey & Patronus are teaming up to bring Patronus' advanced AI evaluators onto the Portkey Gateway — so that you can enforce real-time LLM behavior and take precise actions to secure production.
Portkey Gateway Integrates with 10+ Patronus Evaluators
With this integration, on any of your LLM outputs, you can check for -
- Whether the answer is on-topic to the input question
- Detecting gender or racial bias
- Detecting toxicity
- Multiple evaluations useful in chat scenarios like, checking for conciseness, helpfulness, politeness
- and more!
It's incredibly easy to integrate Patronus with Portkey and start using these checks for your Portkey requests.
- Add Patronus API key to Portkey
- Create Guardrail Checks by selecting the Patronus evaluators you want
- Set up actions on the Guardrails and then add the Guardrail to a request Config
That's it! With this, Portkey will start logging all of the Patronus evaluator results for each request directly.
Check out this documentation for more info.
How Are Guardrails Useful?
When your app hits production scale, there are a lot of little nitty-gritty with LLMs that you will see start to break. More edge cases, more malicious behavior, and overall, less predictability over the LLM behavior itself. Portkey's Gateway & Observability suite helps you debug requests faster, iterate on LLMs faster, and handle LLM API errors, but it doesn't help you tackle the LLM's behavior itself.
The Best Part? It's Open Source
Anybody can write their own custom Guardrail and integrate with the Portkey Gateway easily - including the Patronus one.
Portkey-AIThis is why, we are partnering with the world's best Guardrails platforms like Patronus AI to bring their capabilities on top of Portkey Gateway.
The Patronus is a kind of positive force, a projection of the very things that the dementor feeds upon — hope, happiness, the desire to survive — but it cannot feel despair, as real humans can, so the dementors can’t hurt it.
Fwiw, as Potterheads ourselves, we knew this collaboration was long coming! 🪄
