Portkey’s AI gateway supports making calls to multiple Image models to generate images through a unified API. This notebook showcases the following functionality:
Generating an image through OpenAI
Use the same request to generate an image using Stability AI
Setup a load balance between OpenAI and Stability, with a fallback to OpenAI’s dall-e-2
Cache image requests for super fast loading
This notebook uses the OpenAI SDK to showcase the functionality. We’re using the hosted AI gateway on portkey.ai, but you could swap it for an internally hosted gateway as well.
Copy
Ask AI
# Constants for use later - Please enter your ownPORTKEY_API_KEY="" # Get this from your Portkey AccountOPENAI_API_KEY = "" # Your OpenAI key hereSTABILITY_API_KEY = "" # Add your stability ai API key
Let’s try to make an image generation request to Stability through Portkey. Notice that we’re going to use the OpenAI SDK itself to make calls to Stability AI as well
Copy
Ask AI
from IPython.display import display, Imageimport base64client = OpenAI( api_key=STABILITY_API_KEY, base_url=PORTKEY_GATEWAY_URL, default_headers=createHeaders( provider="stability-ai", api_key=PORTKEY_API_KEY ))# Portkey will automatically convert this request to the format Stability expectsimage = client.images.generate( model="stable-diffusion-v1-6", prompt="Lucy in the sky with diamonds", n=1, size="256x256")# Since stability returns a base64 image string, we can display it like thisimage_bytes = base64.b64decode(image.data[0].b64_json)display(Image(data=image_bytes))
The AI gateway allows us to create routing configurations for better reliability across our requests. Lets take an example where we might want to loadbalance our requests equally between OpenAI’s dall-e-3 and Stability’s stable-diffusion-v1-6 with a overall fallback to dall-e-2This requires us to create a config with a structure like this
Let’s define this using Portkey’s configuration to achieve the same result. You can find more about configs here.
Copy
Ask AI
# It is recommended to create this in the Portkey Config creator, but we're writing the config here to show the processconfig = { "strategy": { "mode": "fallback" }, "targets": [{ "strategy": { "mode": "loadbalance" }, "targets": [{ "provider": "openai", "api_key": OPENAI_API_KEY, },{ "provider": "stability-ai", "api_key": STABILITY_API_KEY, "override_params": {"model": "stable-diffusion-v1-6"} }] },{ "provider": "openai", "api_key": "OPENAI_API_KEY", "override_params": {"model": "dall-e-2"} }]}client = OpenAI( api_key="X", # Not necessary since we''ll pick it up from the config base_url=PORTKEY_GATEWAY_URL, default_headers=createHeaders( config=config, api_key=PORTKEY_API_KEY ))image = client.images.generate( model="dall-e-3", prompt="Lucy in the sky with diamonds", response_format='b64_json', size="1024x1024")# Display the imageimage_bytes = base64.b64decode(image.data[0].b64_json)display(Image(data=image_bytes))
The above image generated will follow your fallback and load balancing configurations making your app very resilient.
The AI gateway also supports caching requests making them extremely fast. We could add cache to the above config and try the requests again.
Copy
Ask AI
# Add simple caching to the config defined aboveconfig["cache"] = {"mode": "simple"}client = OpenAI( api_key="X", # Not necessary since we''ll pick it up from the config base_url=PORTKEY_GATEWAY_URL, default_headers=createHeaders( config=config, api_key=PORTKEY_API_KEY ))image = client.images.generate( model="dall-e-3", prompt="Lucy in the sky with diamonds", response_format='b64_json', size="1024x1024")# Display the imageimage_bytes = base64.b64decode(image.data[0].b64_json)display(Image(data=image_bytes))