Portkey lets you run Bedrock batch jobs without any manual S3 wrangling—simply upload an OpenAI-format .jsonl file and Portkey converts it on-the-fly to the Bedrock format. This is the most efficient way to
  • Test your data with different foundation models
  • Perform A/B testing with different foundation models
  • Perform batch inference with different foundation models
Portkey supports two modes on Bedrock:

​
Before You Start

  1. Portkey API key ($PORTKEY_API_KEY).
  2. Bedrock credentials — either a Portkey Virtual Key or explicit AWS keys (aws_access_key_id, aws_secret_access_key, aws_region, optional aws_session_token).
  3. S3 bucket with read/write access for inputs and outputs.
  4. IAM roles (see Permissions & IAM below).
  5. Optional: a Portkey File (input_file_id) — required only when you set completion_window:"immediate" (Portkey-Batch mode).

​
Using Bedrock Batch API through Portkey

​
Create Batch Job

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="bedrock",
    aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID",
    aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY",
    aws_region="YOUR_AWS_REGION",
    aws_s3_bucket="YOUR_AWS_S3_BUCKET",
    aws_s3_object_key="YOUR_AWS_S3_OBJECT_KEY",
    aws_bedrock_model="YOUR_AWS_BEDROCK_MODEL"
)

start_batch_response = portkey.batches.create(
  input_file_id="file_id", # file id of the input file
  endpoint="endpoint", # ex: /v1/chat/completions
  completion_window="completion_window", # ex: 24h
  metadata={}, # metadata for the batch,
  role_arn="arn:aws:iam::12312:role/BedrockBatchRole", # the role to use for creating the batch job
  model="anthropic.claude-3-5-sonnet-20240620-v1:0", # the model to use for the batch
  output_data_config={
    "s3OutputDataConfig": {
      "s3Uri": "s3://generations-raw/",
      "s3EncryptionKeyId": "arn:aws:kms:us-west-2:517194595696:key/89b483cb-130d-497b-aa37-7db177e7cd32" # this is optional, if you want to use a KMS key to encrypt the output data
    }
  }, # output_data_config is optional, if you want to specify a different output location for the batch job, default is the same as the input file
  job_name="anthropi-requests-test" # optional
)

print(start_batch_response)

​
List Batch Jobs

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="bedrock",
    aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID",
    aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY",
    aws_region="YOUR_AWS_REGION",
)

batches = portkey.batches.list()

print(batches)

​
Get Batch Job Details

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",   
)

batch = portkey.batches.retrieve(batch_id="batch_id")

print(batch)

​
Get Batch Output

curl --location 'https://api.portkey.ai/v1/batches/<batch_id>/output' \
--header 'x-portkey-api-key: <portkey_api_key>' \
--header 'x-portkey-provider: @provider' \

​
List Batch Jobs

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",   
)

batches = portkey.batches.list()

print(batches)

​
Cancel Batch Job

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",   
)

cancel_batch_response = portkey.batches.cancel(batch_id="batch_id")

print(cancel_batch_response)

​
Information about Permissions and IAM Roles

​
Defaults & Limits (Bedrock Native)

PropertyDefaultNotes
completion_window24hFixed by Bedrock.
Job quota50k/daySubject to your AWS account.
Max input file size10 GBAcross all .jsonl objects.
Portkey-Batch mode inherits the Gateway defaults (25 req / 5 s) and retries (3×) per request.

​
Glossary

TermMeaning
Batch JobCollection of asynchronous completions.
Portkey File (input_file_id)File uploaded to Portkey and re-uploaded to the provider for batch processing.
Virtual KeyBedrock credential stored in Portkey; referenced by ID, not secret.
Completion Windowimmediate → Portkey-Batch; 24h → Bedrock native.

​
See Also