Automatic Retries

LLM APIs often have inexplicable failures. With Portkey, you can rescue a substantial number of your requests with our in-built automatic retries feature.

  • Automatic retries are triggered up to 5 times

  • Retries can also be triggered only on specific error codes

  • And each subsequent retry attempt follows exponential backoff strategy to prevent network overload

Enabling Retries

To enable retry, just add the retry param to your config object.

Retry with 5 attempts

    "retry": {
        "attempts": 5
    "virtual_key": "virtual-key-xxx"

Retry only on specific error codes

By default, Portkey triggers retries on the following error codes: [429, 500, 502, 503, 504]

You can change this behaviour by setting the optional on_status_codes param in your retry config and manually inputting the error codes on which rety will be triggered.

  "retry": {
    "attempts": 3,
    "on_status_codes": [ 408, 429, 401 ]
  "virtual_key": "virtual-key-xxx"

If the on_status_codes param is present, retries will be triggered only on the error codes specified in that Config and not on Portkey's default error codes for retries (i.e. [429, 500, 502, 503, 504])

Exponential backoff strategy

Here's how Portkey triggers retries following exponential backoff:

AttemptTime out between requests

Initial Call


Retry 1st attempt

1 second

Retry 2nd attempt

2 seconds

Retry 3rd attempt

4 seconds

Retry 4th attempt

8 seconds

Retry 5th attempt

16 seconds

Last updated