Bring Your Own Guardrails

Portkey supports bringing your own Guardrail using a custom webhook:

Add Webhook as a Guardrail Check and setup any Guardrail Actions along with it. This is useful when you have an existing custom guardrail pipeline in your app and you are sending your LLM inputs or outputs to it for sync or async evaluation. Bring that onto Portkey’s Gateway, make it production-grade, and enforce LLM behavior in real-time, easily.

​

How Custom Webhook Works

In the Guardrail check, Portkey expects your Webhook URL and any headers (including Authorization) you need to send.

​

Headers

Headers here should be a JSON object, like below:

{
    "Authorization":"MY_API_KEY",
    "Content-Type":"Application/JSON"
}

Portkey makes a POST request to your webhook URL and expects two objects in the response: verdict and data.

​

Webhook Response

Portkey expects two objects: verdict and data

​

Here’s a sample webhook response:

{
  "verdict": true,
  "data": {
    "reason": "User passed all security checks",
    "score": 0.95,
    "additionalInfo": {
      "userStatus": "verified",
      "lastCheckTimestamp": "2024-08-21T14:30:00Z"
    }
  }
}

​

Check out the Webhook implementation here:

import { PluginContext, PluginHandler, PluginParameters } from '../types';
import { post } from '../utils';

export const handler: PluginHandler = async (
  context: PluginContext,
  parameters: PluginParameters
) => {
  let error = null;
  let verdict = false;
  let data = null;
  try {
    let url = parameters.webhookURL;
    let headers: Record<string, string=""> = parameters?.headers
      ? JSON.parse(parameters.headers)
      : {};
    ({ verdict, data } = await post(url, context, { headers }, 3000));
  } catch (e: any) {
    delete e.stack;
    error = e;
  }
  return { error, verdict, data };
};
</string,>

Based on the verdict value, the Guardrail Check will PASS or FAIL, and will have subsequent impact on the Guardrail Actions you’ve set.

Head over to the Portkey Discord community if you are building out custom webhooks and need any help!