Portkey Docs

Inference API

Admin API

Changelog ↗

Sign in →

Support

Github

list Run

The type of tool being defined: `code_interpreter`

Code interpreter tool

The maximum number of results the file search tool should output. The default is 20 for gpt-4* models and 5 for gpt-3.5-turbo. This number should be between 1 and 50 inclusive.

Note that the file search tool may output fewer than `max_num_results` results. See the [file search tool documentation](https://platform.openai.com/docs/assistants/tools/file-search/number-of-chunks-returned) for more information.


The type of tool being defined: `file_search`

FileSearch tool

The type of tool being defined: `function`

Function tool

An object describing the expected output of the model. If `json_object` only `function` type `tools` are allowed to be passed to the Run. If `text` the model can return text or any value needed.


Specifies the format that the model must output. Compatible with [GPT-4o](https://platform.openai.com/docs/models/gpt-4o), [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4), and all GPT-3.5 Turbo models since `gpt-3.5-turbo-1106`.

Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.

**Important:** when using JSON mode, you **must** also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.


Controls which (if any) tool is called by the model.
`none` means the model will not call any tools and instead generates a message.
`auto` is the default value and means the model can pick between generating a message or calling one or more tools.
`required` means the model must call one or more tools before responding to the user.
Specifying a particular tool like `{"type": "file_search"}` or `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool.


`none` means the model will not call any tools and instead generates a message. `auto` means the model can pick between generating a message or calling one or more tools. `required` means the model must call one or more tools before responding to the user.


Specifies a tool the model should use. Use to force the model to call a specific tool.

The type of the tool. If type is `function`, the function name must be set

A description of what the function does, used by the model to choose when and how to call the function.

The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://platform.openai.com/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. 

Omitting `parameters` defines a function with an empty parameter list.

Whether to enable [parallel function calling](https://platform.openai.com/docs/guides/function-calling/parallel-function-calling) during tool use.

Usage statistics related to the run. This value will be `null` if the run is not in a terminal state (i.e. `in_progress`, `queued`, etc.).

Number of completion tokens used over the course of the run.

Number of prompt tokens used over the course of the run.

Total number of tokens used (prompt + completion).

Represents an execution run on a [thread](https://platform.openai.com/docs/api-reference/threads).

The ID of the [assistant](https://platform.openai.com/docs/api-reference/assistants) used for execution of this run.

The Unix timestamp (in seconds) for when the run was cancelled.

The Unix timestamp (in seconds) for when the run was completed.

The Unix timestamp (in seconds) for when the run was created.

The Unix timestamp (in seconds) for when the run will expire.

The Unix timestamp (in seconds) for when the run failed.

The identifier, which can be referenced in API endpoints.

Details on why the run is incomplete. Will be `null` if the run is not incomplete.

The reason why the run is incomplete. This will point to which specific token limit was reached over the course of the run.

The instructions that the [assistant](https://platform.openai.com/docs/api-reference/assistants) used for this run.

The last error associated with this run. Will be `null` if there are no errors.

One of `server_error`, `rate_limit_exceeded`, or `invalid_prompt`.

A human-readable description of the error.

The maximum number of completion tokens specified to have been used over the course of the run.


The maximum number of prompt tokens specified to have been used over the course of the run.


The model that the [assistant](https://platform.openai.com/docs/api-reference/assistants) used for this run.

The object type, which is always `thread.run`.

Details on the action required to continue the run. Will be `null` if no action is required.

Details on the tool outputs needed for this run to continue.

For now, this is always `submit_tool_outputs`.

The Unix timestamp (in seconds) for when the run was started.

The status of the run, which can be either `queued`, `in_progress`, `requires_action`, `cancelling`, `cancelled`, `failed`, `completed`, `incomplete`, or `expired`.

The sampling temperature used for this run. If not set, defaults to 1.

The ID of the [thread](https://platform.openai.com/docs/api-reference/threads) that was executed on as a part of this run.

The list of tools that the [assistant](https://platform.openai.com/docs/api-reference/assistants) used for this run.

The nucleus sampling value used for this run. If not set, defaults to 1.

A run on a thread

The run object

The arguments that the model expects you to pass to the function.

The ID of the tool call. This ID must be referenced when you submit the tool outputs in using the [Submit tool outputs to run](https://platform.openai.com/docs/api-reference/runs/submitToolOutputs) endpoint.

The type of tool call the output is required for. For now, this is always `function`.

Controls for how a thread will be truncated prior to the run. Use this to control the intial context window of the run.

The number of most recent messages from the thread when constructing the context for the run.

The truncation strategy to use for the thread. The default is `auto`. If set to `last_messages`, the thread will be truncated to the n most recent messages in the thread. When set to `auto`, messages in the middle of the thread will be dropped to fit the context length of the model, `max_prompt_tokens`.

Thread Truncation Controls

Portkey Key & Virtual Key

Portkey Key & Provider Auth & Provider Name

Portkey Key & Config

Portkey Key & Provider Auth & Provider Name & Custom Host

Returns a list of runs belonging to a thread.

Portkey AI is a comprehensive platform designed to streamline and enhance AI integration for developers and organizations. It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in your Generative AI apps.

What is Portkey?

Integrate Portkey and analyze your first LLM call in 2 minutes!

Make Your First Request

Portkey Features

Portkey connects with all major LLM providers and orchestration frameworks.

Supported LLMs

Google Gemini

Azure OpenAI is a great alternative to accessing the best models including GPT-4 and more in your private environments. Portkey provides complete support for Azure OpenAI.

Azure OpenAI

AWS Bedrock

Google Vertex AI

Ollama

LocalAI

AI21

Integrate Anyscale endpoints with Portkey seamlessly and make your OSS models production-ready

Anyscale

Cerebras

Cohere

Fireworks

Deepinfra

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Deepbricks](https://deepbricks.ai/). 

Deepbricks

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including DeepSeek models. 

DeepSeek

Groq

Hugging Face

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on [Inference.net](https://www.inference.net/). 

Inference.net

Jina AI

Lingyi (01.ai)

Mistral AI

MonsterAPIs provides access to generative AI model APIs at 80% lower costs. Connect to MonsterAPI LLM APIs seamlessly through Portkey's AI gateway.

Monster API

Moonshot

Nomic

Novita AI

OpenRouter

Perplexity AI

Predibase

Reka AI

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [SambaNova AI](https://sambanova.ai/).

SambaNova

Segmind

Stability AI

SiliconFlow

Together AI

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including Voyage AI's embedding and Re-rank endpoints. 

Voyage AI

Workers AI

ZhipuAI / ChatGLM / BigModel

Bring Your Own LLM

Suggest a new integration!

Portkey helps bring your agents to production!

Agents

Use Portkey with Autogen to take your AI Agents to production

Autogen

Use Portkey with Control Flow to take your AI Agents to production

Control Flow

Use Portkey with CrewAI to take your AI Agents to production

CrewAI

Langchain Agents

Use Portkey with LangGraph to take your AI Agents to production

LangGraph Agents

Use Portkey with Llama Agents to take your AI Agents to production

Llama Agents by Llamaindex

Use Portkey with Phidata to take your AI Agents to production

Phidata

You can also use Portkey if you are doing custom agent orchestration!

Bring Your own Agents

Libraries

AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.

Integrate DSPy with Portkey for production-ready LLM pipelines

DSPy

With Portkey, you can confidently take your Instructor pipelines to production and get complete observability over all of your calls + make them reliable - all with a 2 LOC change!

Instructor

Portkey adds core production capabilities to any Langchain app.

Langchain (Python)

Langchain (JS/TS)

Integrate Portkey with LibreChat for enhanced AI capabilities

LibreChat

The **Portkey x LlamaIndex** integration brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** to apps built on LlamaIndex.

LlamaIndex (Python)

Portkey brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** + **versioning** to your **Promptfoo** projects. This document provides an overview of how to leverage the strengths of both the platforms to streamline your AI development workflow.

Promptfoo

Integrate Portkey with Vercel AI SDK for production-ready and reliable AI apps

Vercel

Gain real-time insights, track key metrics, and streamline debugging with our comprehensive observability suite.

Observability (OpenTelemetry)

The Logs section presents a chronological list of all the requests processed through Portkey.

Logs

The **Tracing** capabilities in Portkey empowers you to monitor the lifecycle of your LLM requests in a unified, chronological view.

Tracing

Analytics

Portkey's Feedback APIs provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app.

Feedback

Metadata

Filters

Easily access your Portkey logs data for further analysis and reporting

Logs Export

Budget Limits

Portkey provides the world's fastest multimodal AI gateway that enables working with multiple LLM providers as well as different modalities very easy. The gateway providers features to improve your app's reliability, cost efficiency and accuracy.

AI Gateway

Portkey's Universal API provides a consistent interface to integrate a wide range of modalities (text, vision, audio) and LLMs (hosted OR local) into your apps.

Universal API

This feature is available on all Portkey plans.

Configs

Conditional Routing

Cache (Simple & Semantic)

Fallbacks

LLM APIs often have inexplicable failures. With Portkey, you can rescue a substantial number of your requests with our in-built automatic retries feature. 

Automatic Retries

Load Balancing feature efficiently distributes network traffic across multiple LLMs.

Load Balancing

You can use Portkey's AI gateway to also canary test new models or prompts in different environments. 

Canary Testing

Manage unpredictable LLM latencies effectively with Portkey's **Request Timeouts**.

Request Timeouts

With Portkey's Prompt Library, you can seamlessly create and manage prompts along with all associated model parameters.

Prompt Library

With Prompt Templates, you can seamlessly create and manage your LLM prompts in one place, and deploy them with just an API call.

Prompt Templates

With Prompt Partials, you can save your commonly used templates (which could be your instruction set, data structure explanation, examples etc.) separately from your prompts and flexibly incorporate them wherever required.

Prompt Partials

See how to use Portkey's prompt templates with OpenAI (or any other provider) SDKs

Retrieve Prompts

Advanced Prompting with JSON Mode

Ship to production confidently with Portkey Guardrails on your requests & responses

Guardrails

With the raw Guardrails mode, we let you define your Guardrail checks & actions however you want, directly in code.

Portkey Endpoints

Provider Endpoints

list Run

Authorizations

Path Parameters

Query Parameters

Response