You are likely familiar with how to make an API call to GPT4 for chat completions.
messages
and model
arguments to get us a response. If you have tried one before, the following code snippet should look familiar. That’s because Portkey Client SDK follows the same signature as OpenAI’s.
portkey
variable — virtualKey
. Portkey securely stores API keys of LLM providers in a vault and substitutes them at runtime in your requests. These unique identifiers to your API keys are called Virtual Keys. For more information, see the docs.
With basics out of our way, let’s jump into applying what we set out to do in the first place with the AI gateway — To automatically retry our request when we hit rate-limits (429 status codes).
ID
:
Try it out now!
The Configs saved will appear as a row item on the Configs page. The ID
is important as it is referenced in our calls through the AI gateway.
portkey
is instantiated.
That’s right! One line of code, and all the request from your apps now inherit Gateway Configs and demonstrate automatic retries.
Let’s take a look at the code snippet:
x-portkey-config
.
baseURL
and necessary headers as follows:
config
parameter as an argument.
config
argument as seperate objects right at the time of chat completions call instead of Portkey({..})
instantiation.