Manage unpredictable LLM latencies effectively with Portkey’s Request Timeouts.
integer)
x-portkey-request-timeout
header.
azure-openai
), there is a timeout override, set at 10s, which will be applied only to this targetazure-openai
), the top strategy-level timeout of 2s will be appliedon_status_codes
parameter, ensuring robust handling of these scenarios.
gpt-4
often have sub-10-second response times