AWS Bedrock
Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including models hosted on AWS Bedrock.
With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a virtual key system.
Portkey SDK Integration with AWS Bedrock
Portkey provides a consistent API to interact with models from various providers. To integrate Bedrock with Portkey:
1. Install the Portkey SDK
Add the Portkey SDK to your application to interact with Anthropic's API through Portkey's gateway.
2. Initialize Portkey with the Virtual Key
Set up Portkey with your virtual key as part of the initialization configuration. You can create a virtual key for AWS Bedrock in the UI.
You can navigate to the Virtual Keys tab of your Portkey account and create a Key for AWS Bedrock that'll make it easier to make calls. You'll need the AWS Secret Access Key
and AWS Access Key Id
to create the virtual key.
Using Virtual Key with AWS STS
If you're using AWS Security Token Service, you can pass your aws_session_token
along with the Virtual key:
Not using Virtual Keys?
3. Invoke Chat Completions with AWS bedrock
Use the Portkey instance to send requests to Anthropic. You can also override the virtual key directly in the API call if needed.
Using Vision Models
Portkey's multimodal Gateway fully supports Bedrock's vision models anthropic.claude-3-sonnet
, anthropic.claude-3-haiku
, and anthropic.claude-3-opus
For more info, check out this guide:
pageVisionManaging AWS Bedrock Prompts
You can manage all prompts to AWS bedrock in the Prompt Library. All the current models of Anthropic are supported and you can easily start testing different prompts.
Once you're ready with your prompt, you can use the portkey.prompts.completions.create
interface to use the prompt in your application.
Making Requests without Virtual Keys
If you do not want to add your AWS details to Portkey vault, you can also directly pass them while instantiating the Portkey client.
Mapping the Bedrock Details
Node SDK | Python SDK | REST Headers |
---|---|---|
awsAccessKeyId | aws_access_key_id | x-portkey-aws-session-token |
awsSecretAccessKey | aws_secret_access_key | x-portkey-aws-secret-access-key |
awsRegion | aws_region | x-portkey-aws-region |
awsSessionToken | aws_session_token | x-portkey-aws-session-token |
Example
Supported Models
Model Name | Model String to Use in API calls |
---|---|
Amazon Titan Text Express |
|
Amazon Titan Lite |
|
Amazon Titan Embeddings |
|
Anthropic Claude 1 |
|
Anthropic Claude 2 |
|
Anthropic Claude 2.1 |
|
Anthropic Claude Instant 1 |
|
Anthropic Claude 3 Sonnet |
|
AI21 J2 Mid |
|
AI21 J2 Ultra |
|
Cohere Command |
|
Cohere Command Light |
|
Cohere Embeddings (English) |
|
Cohere Embeddings (multilingual) |
|
Meta Llama2 13B Chat |
|
Meta Llama2 70B Chat |
|
Stability Image Generation models | All |
Mistral 7B Instruct |
|
Mixtral 8X7B Instruct |
|
Mistral Large |
|
Next Steps
The complete list of features supported in the SDK are available on the link below.
pagePortkey SDK ClientYou'll find more information in the relevant sections:
Last updated