Google Vertex AI
Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including Google Vertex AI.
With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a virtual key system.
Portkey SDK Integration with Google Vertex AI
Portkey provides a consistent API to interact with models from various providers. To integrate Google Vertex AI with Portkey:
1. Install the Portkey SDK
Add the Portkey SDK to your application to interact with Google Vertex AI API through Portkey's gateway.
2. Initialize Portkey with the Virtual Key
Set up Portkey with your virtual key as part of the initialization configuration. You can create a virtual key for Vertex AI in the UI.
If you do not want to add your Vertex AI details to Portkey vault, you can directly pass them while instantiating the Portkey client. More on that here.
3. Invoke Chat Completions with Vertex AI and Gemini
Use the Portkey instance to send requests to Gemini models hosted on Vertex AI. You can also override the virtual key directly in the API call if needed.
Vertex AI uses OAuth2 to authenticate its requests, so you need to send the access token additionally along with the request.
Function Calling
Portkey supports function calling mode on Google's Gemini Models. Explore this ⬇️ Cookbook for a deep dive and examples:
pageFunction CallingManaging Vertex AI Prompts
You can manage all prompts to Google Gemini in the Prompt Library. All the models in the model garden are supported and you can easily start testing different prompts.
Once you're ready with your prompt, you can use the portkey.prompts.completions.create
interface to use the prompt in your application.
Making Requests Without Virtual Keys
You can also pass your Vertex AI details & secrets directly without using the Virtual Keys in Portkey.
Vertex AI expects a region
, a project ID
and the access token
in the request for a successful completion request. This is how you can specify these fields directly in your requests:
For further questions on custom Vertex AI deployments or fine-grained access tokens, reach out to us on [email protected]
Next Steps
The complete list of features supported in the SDK are available on the link below.
pagePortkey SDK ClientYou'll find more information in the relevant sections:
Last updated