> ## Documentation Index > Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # AWS Bedrock Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including models hosted on AWS Bedrock. With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through [Model Catalog](/product/model-catalog). ## Portkey SDK Integration with AWS Bedrock Portkey provides a consistent API to interact with models from various providers. To integrate Bedrock with Portkey: ### 1. Install the Portkey SDK Add the Portkey SDK to your application to interact with Anthropic's API through Portkey's gateway. ```sh theme={"system"} npm install --save portkey-ai ``` ```sh theme={"system"} pip install portkey-ai ``` ### 2. Initialize Portkey with the Bedrock Provider There are two ways to integrate AWS Bedrock with Portkey:
Use your `AWS Secret Access Key`, `AWS Access Key Id`, and `AWS Region` to create your AI Provider on Portkey's app.

[**Integration Guide**](/integrations/llms/aws-bedrock#how-to-find-your-aws-credentials)
Take your `AWS Assumed Role ARN` and `AWS Region` to create your Bedrock provider.


[**Integration Guide**](/product/model-catalog/connect-bedrock-with-amazon-assumed-role) ```js theme={"system"} import Portkey from 'portkey-ai' const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"] provider:"@PROVIDER" // Your Bedrock Provider Slug }) ``` ```python theme={"system"} from portkey_ai import Portkey portkey = Portkey( api_key="PORTKEY_API_KEY", # Replace with your Portkey API key provider="@PROVIDER" # Replace with Your Bedrock Provider Slug ) ``` #### Using Bedrock Provider with AWS STS If you're using [AWS Security Token Service](https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html), you can pass your `aws_session_token` along with the AI Provider slug: ```js theme={"system"} import Portkey from 'portkey-ai' const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"] provider:"@PROVIDER" // Your Bedrock Provider Slug, aws_session_token: "" }) ``` ```python theme={"system"} from portkey_ai import Portkey portkey = Portkey( api_key="PORTKEY_API_KEY", # Replace with your Portkey API key provider="@PROVIDER" # Replace with your Provider Slug for Bedrock, aws_session_token="" ) ``` #### Not using Bedrock Provider from Model Catalog? [Check out this example on how you can directly use your AWS details to make a Bedrock request through Portkey.](/integrations/llms/bedrock/aws-bedrock#making-requests-without-using-portkey%E2%80%99s-model-catalog) ### **3. Invoke Chat Completions with AWS bedrock** Use the Portkey instance to send requests to Anthropic. You can also override the provider slug directly in the API call if needed. ```js theme={"system"} const chatCompletion = await portkey.chat.completions.create({ messages: [{ role: 'user', content: 'Say this is a test' }], model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0', max_tokens: 250 // Required field for Anthropic }); console.log(chatCompletion.choices); ``` ```python theme={"system"} completion = portkey.chat.completions.create( messages= [{ "role": 'user', "content": 'Say this is a test' }], model= 'us.anthropic.claude-3-7-sonnet-20250219-v1:0', max_tokens=250 # Required field for Anthropic ) print(completion.choices) ``` ## Using the /messages Route with Bedrock Models Access Bedrock's Claude models through Anthropic's native`/messages` endpoint using Portkey's SDK or Anthropic's SDK. This route only works with Claude models on Bedrock. For other models, use the standard OpenAI compliant endpoint. ```sh theme={"system"} curl --location 'https://api.portkey.ai/v1/messages' \ --header 'x-portkey-provider: @your-bedrock-provider' \ --header 'Content-Type: application/json' \ --header 'x-portkey-api-key: YOUR_PORTKEY_API_KEY' \ --data '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 250, "messages": [ { "role": "user", "content": "Hello, Claude" } ] }' ``` ```python theme={"system"} Coming Soon! ``` ```javascript theme={"system"} Coming Soon! ``` ```python Anthropic Python SDK theme={"system"} import anthropic client = anthropic.Anthropic( api_key="dummy", # we will use portkey's provider slug default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"}, base_url="https://api.portkey.ai" ) message = client.messages.create( model="@your-provider-slug/your-model-name", max_tokens=250, messages=[ {"role": "user", "content": "Hello, Claude"} ], ) print(message.content) ``` ```typescript Anthropic TS SDK theme={"system"} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ apiKey: 'dummy', // we will use portkey's provider slug baseURL: "https://api.portkey.ai/v1", defaultHeaders: { "x-portkey-api-key": "YOUR_PORTKEY_API_KEY" } }); const msg = await anthropic.messages.create({ model: "@your-provider-slug/your-model-name", max_tokens: 1024, messages: [{ role: "user", content: "Hello, Claude" }], }); console.log(msg); ``` Portkey supports the [AWS Bedrock CountTokens API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CountTokens.html) to estimate token usage before sending requests. Check out the count-tokens guide for more details. ## Using Vision Models Portkey's multimodal Gateway fully supports Bedrock's vision models `anthropic.claude-3-sonnet`, `anthropic.claude-3-haiku`, and `anthropic.claude-3-opus` For more info, check out this guide: [Vision](/product/ai-gateway/multimodal-capabilities/vision) ### Using S3 URIs in Chat Completions Portkey supports S3 URIs directly in the `image_url` field for chat completions. This allows you to reference files stored in S3 without needing to convert them to base64. When using S3 URIs, the `mime_type` parameter is **required**. The gateway cannot infer the MIME type from an S3 URI, so you must specify it explicitly. ```python theme={"system"} from portkey_ai import Portkey portkey = Portkey( api_key="PORTKEY_API_KEY", provider="@PROVIDER" ) response = portkey.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=1024, messages=[ { "role": "user", "content": [ { "type": "text", "text": "What's in this image?" }, { "type": "image_url", "image_url": { "url": "s3://your-bucket/path/to/image.png", "mime_type": "image/png", # Required for S3 URIs "bucketOwner": "123456789012" # Optional: AWS account ID of the bucket owner } } ] } ] ) ``` ```javascript theme={"system"} import Portkey from 'portkey-ai'; const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY", provider: "@PROVIDER" }); const response = await portkey.chat.completions.create({ model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens: 1024, messages: [ { role: "user", content: [ { type: "text", text: "What's in this image?" }, { type: "image_url", image_url: { url: "s3://your-bucket/path/to/image.png", mime_type: "image/png", // Required for S3 URIs bucketOwner: "123456789012" // Optional: AWS account ID of the bucket owner } } ] } ] }); ``` ```sh theme={"system"} curl "https://api.portkey.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: @your-bedrock-provider-slug" \ -d '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 1024, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "What is in this image?" }, { "type": "image_url", "image_url": { "url": "s3://your-bucket/path/to/image.png", "mime_type": "image/png", "bucketOwner": "123456789012" } } ] } ] }' ``` #### S3 URI Parameters | Parameter | Type | Required | Description | | ------------- | ------ | -------- | ----------------------------------------------------------------------------- | | `url` | string | Yes | The S3 URI (e.g., `s3://bucket-name/path/to/file.png`) | | `mime_type` | string | Yes | The MIME type of the file (see supported formats below) | | `bucketOwner` | string | No | The AWS account ID of the bucket owner. Required for cross-account S3 access. | #### Supported Image Formats for S3 URIs | Format | MIME Type | | ------ | ------------ | | PNG | `image/png` | | JPEG | `image/jpeg` | | GIF | `image/gif` | | WebP | `image/webp` | Ensure your Bedrock IAM role has the necessary S3 permissions (`s3:GetObject`) to access the referenced objects. ## Extended Thinking (Reasoning Models) (Beta) The assistants thinking response is returned in the `response_chunk.choices[0].delta.content_blocks` array, not the `response.choices[0].message.content` string. Models like `us.anthropic.claude-3-7-sonnet-20250219-v1:0` support [extended thinking](https://aws.amazon.com/blogs/aws/anthropics-claude-3-7-sonnet-the-first-hybrid-reasoning-model-is-now-available-in-amazon-bedrock/). This is similar to openai thinking, but you get the model's reasoning as it processes the request as well. Note that you will have to set [`strict_open_ai_compliance=False`](/product/ai-gateway/strict-open-ai-compliance) in the headers to use this feature. ### Using `reasoning_effort` parameter You can also use the OpenAI-compatible `reasoning_effort` parameter as a drop-in alternative to `thinking.budget_tokens`: ```python theme={"system"} response = portkey.chat.completions.create( model="us.anthropic.claude-sonnet-4-6-20250929-v1:0", max_tokens=3000, reasoning_effort="medium", # Options: "none", "minimal", "low", "medium", "high" messages=[{"role": "user", "content": "Explain quantum computing"}] ) ``` * **Claude Opus 4.6 / Sonnet 4.6**: `reasoning_effort` enables [adaptive thinking](https://platform.claude.com/docs/en/build-with-claude/effort#effort-with-extended-thinking) (`thinking.type = "adaptive"`) — Claude decides the thinking budget automatically. * **Other Claude reasoning models**: `reasoning_effort` is converted to `thinking.budget_tokens` as a ratio of `max_tokens` (minimal 10%, low 20%, medium 50%, high 80%), with a minimum of 1024 tokens. `max_tokens` must be greater than 1024. * `reasoning_effort: "none"` disables extended thinking. * An explicit `thinking` object in the request always takes precedence over `reasoning_effort`. ### Single turn conversation ```py Python theme={"system"} from portkey_ai import Portkey # Initialize the Portkey client portkey = Portkey( api_key="PORTKEY_API_KEY", # Replace with your Portkey API key provider="@PROVIDER", strict_openai_compliance=False ) # Create the request response = portkey.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) # in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array # response = portkey.chat.completions.create( # ...same config as above but with stream: true # ) # for chunk in response: # if chunk.choices[0].delta: # content_blocks = chunk.choices[0].delta.get("content_blocks") # if content_blocks is not None: # for content_block in content_blocks: # print(content_block) ``` ```ts NodeJS theme={"system"} import Portkey from 'portkey-ai'; // Initialize the Portkey client const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY", // Replace with your Portkey API key provider:"@PROVIDER", // Add your bedrock's Provider slug from model catalog strictOpenAICompliance: false }); // Generate a chat completion async function getChatCompletionFunctions() { const response = await portkey.chat.completions.create({ model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens: 3000, thinking: { type: "enabled", budget_tokens: 2030 }, stream: true, messages: [ { role: "user", content: [ { type: "text", text: "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }); console.log(response); // in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array // const response = await portkey.chat.completions.create({ // ...same config as above but with stream: true // }); // for await (const chunk of response) { // if (chunk.choices[0].delta?.content_blocks) { // for (const contentBlock of chunk.choices[0].delta.content_blocks) { // console.log(contentBlock); // } // } // } } // Call the function getChatCompletionFunctions(); ``` ```js OpenAI NodeJS theme={"system"} import OpenAI from 'openai'; // We're using the v4 SDK import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai' const openai = new OpenAI({ apiKey: 'dummy', // defaults to process.env["OPENAI_API_KEY"], baseURL: PORTKEY_GATEWAY_URL, defaultHeaders: createHeaders({ provider: "@bedrock", // your portkey bedrock provider slug from model catalog apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"] strictOpenAICompliance: false }) }); // Generate a chat completion with streaming async function getChatCompletionFunctions(){ const response = await openai.chat.completions.create({ model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens: 3000, thinking: { type: "enabled", budget_tokens: 2030 }, stream: true, messages: [ { role: "user", content: [ { type: "text", text: "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ], }); console.log(response) // in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array // const response = await openai.chat.completions.create({ // ...same config as above but with stream: true // }); // for await (const chunk of response) { // if (chunk.choices[0].delta?.content_blocks) { // for (const contentBlock of chunk.choices[0].delta.content_blocks) { // console.log(contentBlock); // } // } // } } await getChatCompletionFunctions(); ``` ```py OpenAI Python theme={"system"} from openai import OpenAI from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders openai = OpenAI( api_key='dummt', base_url=PORTKEY_GATEWAY_URL, default_headers=createHeaders( provider="@bedrock", # your bedrock provider slug from model catalog api_key="PORTKEY_API_KEY", strict_openai_compliance=False ) ) response = openai.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] ) print(response) ``` ```sh cURL theme={"system"} curl "https://api.portkey.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: @your-bedrock-provider-slug" \ -H "x-portkey-strict-openai-compliance: false" \ -d '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] } ] }' ``` ### Multi turn conversation ```py Python theme={"system"} from portkey_ai import Portkey # Initialize the Portkey client portkey = Portkey( api_key="PORTKEY_API_KEY", # Replace with your Portkey API key provider="@PROVIDER", strict_openai_compliance=False ) # Create the request response = portkey.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```ts NodeJS theme={"system"} import Portkey from 'portkey-ai'; // Initialize the Portkey client const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY", // Replace with your Portkey API key provider:"@PROVIDER", // Add your bedrock's Provider Slug from model catalog strictOpenAICompliance: false }); // Generate a chat completion async function getChatCompletionFunctions() { const response = await portkey.chat.completions.create({ model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens: 3000, thinking: { type: "enabled", budget_tokens: 2030 }, stream: true, messages: [ { role: "user", content: [ { type: "text", text: "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { role: "assistant", content: [ { type: "thinking", thinking: "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { role: "user", content: "thanks that's good to know, how about to chennai?" } ] }); console.log(response); } // Call the function getChatCompletionFunctions(); ``` ```js OpenAI NodeJS theme={"system"} import OpenAI from 'openai'; // We're using the v4 SDK import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai' const openai = new OpenAI({ apiKey: 'dummy', // we are using provider from portkey baseURL: PORTKEY_GATEWAY_URL, defaultHeaders: createHeaders({ provider: "@bedrock", // Replace with your bedrock provider slug from model catalog apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"] strictOpenAICompliance: false }) }); // Generate a chat completion with streaming async function getChatCompletionFunctions(){ const response = await openai.chat.completions.create({ model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens: 3000, thinking: { type: "enabled", budget_tokens: 2030 }, stream: true, messages: [ { role: "user", content: [ { type: "text", text: "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { role: "assistant", content: [ { type: "thinking", thinking: "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { role: "user", content: "thanks that's good to know, how about to chennai?" } ], }); console.log(response) } await getChatCompletionFunctions(); ``` ```py OpenAI Python theme={"system"} from openai import OpenAI from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders openai = OpenAI( api_key='dummy', # we are using provider from portkey base_url=PORTKEY_GATEWAY_URL, default_headers=createHeaders( provider="@bedrock-slug", # Replace with your bedrock provider slug from model catalog api_key="PORTKEY_API_KEY", strict_openai_compliance=False ) ) response = openai.chat.completions.create( model="us.anthropic.claude-3-7-sonnet-20250219-v1:0", max_tokens=3000, thinking={ "type": "enabled", "budget_tokens": 2030 }, stream=True, messages=[ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] ) print(response) ``` ```sh cURL theme={"system"} curl "https://api.portkey.ai/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: @your-bedrock-provider-slug" \ -H "x-portkey-strict-openai-compliance: false" \ -d '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 3000, "thinking": { "type": "enabled", "budget_tokens": 2030 }, "stream": true, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?" } ] }, { "role": "assistant", "content": [ { "type": "thinking", "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.", "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE=" } ] }, { "role": "user", "content": "thanks that's good to know, how about to chennai?" } ] }' ``` ## Inference Profiles [Inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.html) are a resource in Amazon Bedrock that define a model and one or more Regions to which the inference profile can route model invocation requests. To use inference profiles, your IAM role needs to additionally have the following permissions: ```json theme={"system"} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:GetInferenceProfile" ], "Resource": [ "arn:aws:bedrock:*:*:inference-profile/*", "arn:aws:bedrock:*:*:application-inference-profile/*" ] } ] } ``` This is a pre-requisite for using inference profiles, as the gateway needs to fetch the foundation model to process the request. For reference, see the following documentation: [https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html) ## Bedrock Guardrails You can use Bedrock guardrails directly in your chat completions requests to add content filtering and safety measures. Guardrails help ensure that model responses adhere to your specific safety and content policies. We recommend using guardrails through the Portkey UI for easier management and configuration. You can learn more about guardrails [here](/integrations/guardrails/bedrock-guardrails). ### Using Guardrails in Chat Completions To enable guardrails, include the `guardrailConfig` parameter in your request: ```js theme={"system"} const chatCompletion = await portkey.chat.completions.create({ messages: [{ role: 'user', content: 'Say this is a test' }], model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0', max_tokens: 250, guardrailConfig: { guardrailIdentifier: "your-guardrail-id", guardrailVersion: "DRAFT", // or specific version number trace: "enabled" // optional: "enabled" or "disabled" } }); ``` ```python theme={"system"} completion = portkey.chat.completions.create( messages=[{ "role": 'user', "content": 'Say this is a test' }], model='us.anthropic.claude-3-7-sonnet-20250219-v1:0', max_tokens=250, guardrail_config={ # Note: snake_case also supported "guardrailIdentifier": "your-guardrail-id", "guardrailVersion": "DRAFT", "trace": "enabled" } ) ``` ```sh theme={"system"} curl https://api.portkey.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: @your-bedrock-provider" \ -d '{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "max_tokens": 250, "messages": [{"role": "user","content": "Say this is a test"}], "guardrailConfig": { "guardrailIdentifier": "your-guardrail-id", "guardrailVersion": "DRAFT", "trace": "enabled" } }' ``` ### Guardrail Configuration Parameters | Parameter | Type | Required | Description | | --------------------- | ------ | -------- | ----------------------------------------------------------------------------------------------- | | `guardrailIdentifier` | string | Yes | The unique identifier of your Bedrock guardrail | | `guardrailVersion` | string | Yes | Version of the guardrail (`"DRAFT"` for the latest draft version, or a specific version number) | | `trace` | string | No | Controls trace generation (`"enabled"` or `"disabled"`) | Both `guardrailConfig` (camelCase) and `guardrail_config` (snake\_case) parameter names are supported for compatibility. When a guardrail is triggered, the response will include a `guardrail_intervened` stop reason. You can access detailed trace information if tracing is enabled. ## Bedrock Converse API Portkey uses the [AWS Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) internally for making chat completions requests. If you need to pass [additional input fields or parameters](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html#API_runtime_Converse_RequestSyntax) like `anthropic_beta`, `top_k`, `frequency_penalty` etc. that are specific to a model, you can pass it with this key: ```json theme={"system"} "additionalModelRequestFields": { "frequency_penalty": 0.4 } ``` If you require the model to [respond with certain fields](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html#API_runtime_Converse_RequestSyntax) that are specific to a model, you need to pass this key: ```json theme={"system"} "additionalModelResponseFieldPaths": [ "/stop_sequence" ] ``` ## Managing AWS Bedrock Prompts You can manage all prompts to AWS bedrock in the [Prompt Library](/product/prompt-library). All the current models of Anthropic are supported and you can easily start testing different prompts. Once you're ready with your prompt, you can use the `portkey.prompts.completions.create` interface to use the prompt in your application. ## Making Requests without using Portkey's Model Catalog If you do not want to add your AWS details to Portkey vault, you can also directly pass them while instantiating the Portkey client. ### Mapping the Bedrock Details | Node SDK | Python SDK | REST Headers | | ------------------ | ------------------------ | ------------------------------- | | awsAccessKeyId | aws\_access\_key\_id | x-portkey-aws-access-key-id | | awsSecretAccessKey | aws\_secret\_access\_key | x-portkey-aws-secret-access-key | | awsRegion | aws\_region | x-portkey-aws-region | | awsSessionToken | aws\_session\_token | x-portkey-aws-session-token | ### Example ```js theme={"system"} import Portkey from 'portkey-ai' const portkey = new Portkey({ apiKey: "PORTKEY_API_KEY", provider: "bedrock", awsAccessKeyId: "AWS_ACCESS_KEY_ID", awsSecretAccessKey: "AWS_SECRET_ACCESS_KEY", awsRegion: "us-east-1", awsSessionToken: "AWS_SESSION_TOKEN" }) ``` ```python theme={"system"} from portkey_ai import Portkey client = Portkey( api_key="PORTKEY_API_KEY", provider="bedrock", aws_access_key_id="", aws_secret_access_key="", aws_region="us-east-1", aws_session_token="" ) ``` ```sh theme={"system"} curl https://api.portkey.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: bedrock" \ -H "x-portkey-aws-access-key-id: $AWS_ACCESS_KEY_ID" \ -H "x-portkey-aws-secret-access-key: $AWS_SECRET_ACCESS_KEY" \ -H "x-portkey-aws-region: $AWS_REGION" \ -H "x-portkey-aws-session-token: $AWS_TOKEN" \ -d '{ "model": "gpt-4o", "messages": [{"role": "user","content": "Hello!"}] }' ``` *** ## Using AWS PrivateLink for Bedrock \[Self Hosted Enterprise] Though using assumed role is in itself enough for enterprise security. You can additional configure AWS [PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html) for Bedrock to ensure that your requests are not traversed outside your VPC. * Create a private link between the VPC you've deployed Portkey and AWS Bedrock (the endpoint is in most cases `https://bedrock.{your_region}.amazonaws.com`). * When configuring your integration on portkey, simply configure the `custom host` option to point to your VPC endpoint for the private link. ## AWS GovCloud (US) Integration is identical to standard Bedrock. Only the endpoint changes — set a Custom Host for your region. ### Steps 1. In Portkey, create or edit your Bedrock provider. 2. Open "Advanced Options". 3. Set "Custom Host" to your GovCloud Bedrock endpoint: * `https://bedrock.us-gov-east-1.amazonaws.com` * `https://bedrock.us-gov-west-1.amazonaws.com` * and more similarly... 4. Save and use normally in SDKs and via the gateway. Configure Custom Host for AWS GovCloud ### Notes * For the complete list of Bedrock endpoints, see the official AWS reference: [Amazon Bedrock endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/bedrock.html). * For FIPS-compliant endpoints and additional compliance information, see: [FIPS Endpoints by Service](https://aws.amazon.com/compliance/fips/#FIPS_Endpoints_by_Service). ## Supported Models *** ## How to Find Your AWS Credentials [Navigate here in the AWS Management Console](https://us-east-1.console.aws.amazon.com/iam/home#/security%5Fcredentials) to obtain your **AWS Access Key ID** and **AWS** **Secret Access Key.** * In the console, you'll find the '**Access keys'** section. Click on '**Create access key**'. * Copy the `Secret Access Key` once it is generated, and you can view the `Access Key ID` along with it. acess * On the same [page](https://us-east-1.console.aws.amazon.com/iam/home#/security%5Fcredentials) under the '**Access keys'** section, where you created your Secret Access key, you will also find your **Access Key ID.** retrieve * And lastly, get Your `AWS Region` from the Home Page of[ AWS Bedrock](https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/overview) as shown in the image below. bedrock *** ## Next Steps The complete list of features supported in the SDK are available on the link below. You'll find more information in the relevant sections: 1. [Add metadata to your requests](/product/observability/metadata) 2. [Add gateway configs to your Bedrock requests](/product/ai-gateway/configs) 3. [Tracing Bedrock requests](/product/observability/traces) 4. [Setup a fallback from OpenAI to Beckrock APIs](/product/ai-gateway/fallbacks)