paid plan
, route their request to a custom fine-tuned model
fastest
, route to gpt-4o-mini
, if smartest
, route to openai o1
EU resident
, call an EU hosted model
temperature
parameter is above 0.7
, route to a more creative modeltesting environment
with a llm-pass-through
flag, route it to the cheapest model
fallback
and loadbalance
). To use it in your app,
conditional
config in Portkey UI.config
param.conditional
Configconditional
config looks (along with its simpler, tree view).
strategy.mode
: Set to conditional
strategy.conditions
: Query conditions with rules applied on metadata values or request parameters along with which target to call when the condition passesstrategy.default
: The default target name to call when none of the conditions passtargets
: Array of target objects with unique names
and provider details. These target names are referenced in the conditions
objects above.conditions
and default
are required params for the conditional
strategy.conditions
Objectconditions
are where you will actually write the routing rules. Here’s a sample condition
object:
query
: Write the exact rule for checking metadata values or request parameters
then
: Define which target to call if the query PASSES
Operator | Description |
---|---|
$eq | Equals |
$ne | Not equals |
$in | In array |
$nin | Not in array |
$regex | Match the regex |
$gt | Greater than |
$gte | Greater than or equal to |
$lt | Less than |
$lte | Less than or equal to |
$and
: All conditions must be true$or
: At least one condition must be true$and
, $or
operators)default
target name is calledconditions
and the Config structure described above, you can create your Config in Portkey UI, and save it to get Config ID. The UI also helps you autofill and autoformat your Config.
model
, temperature
, or max_tokens
model
, temperature
, max_tokens
, etc.).
model
parameter, allowing you to use aliases instead of specific model names:
model
in your request for routing to work properly.user_plan
metadata sent along request. If the user is on a paid
plan, we route to a finetuned-gpt4
model, otherwise we route to a base-gpt4
model.metadata.user_plan
and params.model
.User-Based Routing
Model Selection & Parameter Routing
Application Features & Testing
Using Conditional Router with Guardrails
model
, temperature
, top_p
, frequency_penalty
, presence_penalty
, max_tokens
, and many others.