Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

Imagine usecases where you want to redact PII, remove some requests and redact responses when making a bulk batch inference call to providers like OpenAI, Azure OpenAI, Bedrock, Vertex AI, etc. Portkey supports running guardrails on provider batch requests. You pass guardrails through a config, and Portkey applies them asynchronously — pre-processing inputs before sending to the provider, and post-processing outputs after the provider completes.

​
How It Works

1

Upload a batch input file

Upload a JSONL file in OpenAI chat completions format to Portkey.
curl --location 'https://api.portkey.ai/v1/files' \
  --header 'x-portkey-api-key: {{PORTKEY_API_KEY}}' \
  --header 'x-portkey-provider: portkey' \
  --form 'file=@"/path/to/batch_requests.jsonl"' \
  --form 'purpose="batch"'
Each line in the file should be a valid chat completions request. Download an example JSONL file.The response returns a Portkey file ID (pk_file_*):
{
  "object": "file",
  "id": "pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca",
  "purpose": "batch",
  "filename": "d2d44065-0f52-4780-a216-eb0122a25d78/pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca.jsonl",
  "bytes": 26204,
  "created_at": 1777975975402,
  "status": "processed",
  "status_details": null
}
2

Create a config with guardrails

Guardrails are the only supported way to pass guardrail checks to a provider batch request. Define them in a Portkey config here:
{
  "retry": {
    "attempts": 3
  },
  "input_guardrails": ["pg-sync-g-0663db"],
  "output_guardrails": ["pg-mutato-9760bc"]
}
Save this config in the Portkey dashboard and note the config ID (e.g., pc-sync-g-1009a3).
3

Create a provider batch

Reference your uploaded file ID and config when creating the batch:
curl --location 'https://api.portkey.ai/v1/batches' \
  --header 'x-portkey-api-key: {{PORTKEY_API_KEY}}' \
  --header 'x-portkey-provider: portkey' \
  --header 'Content-Type: application/json' \
  --data '{
      "input_file_id": "pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca",
      "endpoint": "/v1/chat/completions",
      "completion_window": "24h",
      "model": "gpt-3.5-turbo",
      "portkey_options": {
          "x-portkey-config": "pc-sync-g-1009a3",
          "x-portkey-virtual-key": "openai-virtual-key"
      }
  }'
The response contains a Portkey batch ID (pk_batch_*) and the initial guardrail results in hook_results:
{
  "id": "pk_batch_cd7f0e97-37b6-48cb-abd7-1bb699300585",
  "object": "batch",
  "status": "validating",
  "input_file_id": "pk_file_1db86eb1-22b5-4440-8810-e45ddd1e5aca",
  "hook_results": {
    "before_request_hooks": [],
    "after_request_hooks": [
      {
        "verdict": true,
        "id": "portkey-dataservice",
        "type": "guardrail",
        "deny": false
      }
    ]
  }
}
4

Poll for batch status

Poll the batch to track progress. Once input guardrails are applied, the input_file_id is replaced with a new processed file. Use include_provider_details=true to see full upstream details:
curl --location 'https://api.portkey.ai/v1/batches/pk_batch_cd7f0e97-37b6-48cb-abd7-1bb699300585?include_provider_details=true' \
  --header 'x-portkey-api-key: {{PORTKEY_API_KEY}}'
If your config includes output_guardrails, the batch enters an intermediate processing_post_request_guardrails state after the upstream provider finishes. Once complete, output_file_id reflects the final guardrail-processed output file.
5

Download the output file

Once the batch status is completed, retrieve the output using the Portkey-exclusive file content endpoint:
curl --location 'https://api.portkey.ai/v1/files/{{OUTPUT_FILE_ID}}/content' \
  --header 'x-portkey-api-key: {{PORTKEY_API_KEY}}' \
  --header 'x-portkey-provider: portkey'

​
Batch Lifecycle with Guardrails

StateDescription
validatingBatch created; input guardrails being evaluated
in_progressInput guardrails passed; provider is processing requests
processing_post_request_guardrailsProvider finished; output guardrails being evaluated
completedAll guardrails applied; output file ready
The input_file_id in the batch response is updated after input guardrails process the file — the new ID points to the transformed input sent to the upstream provider.
Last modified on May 19, 2026