Portkey Docs

Integrations

Inference API

Admin API

Cookbook

Changelog

Sign in →

Support

Get Demo

Github

List Fine-tuning Jobs

The type of the integration being enabled for the fine-tuning job

The settings for your integration with Weights and Biases. This payload specifies the project that
metrics will be sent to. Optionally, you can set an explicit display name for your run, add tags
to your run, and set a default entity (team, username, etc) to be associated with your run.


The entity to use for the run. This allows you to set the team or username of the WandB user that you would
like associated with the run. If not set, the default entity for the registered WandB API key is used.


A display name to set for the run. If not set, we will use the Job ID as the name.


The name of the project that the new run will be created under.


Fine-Tuning Job Integration

The `fine_tuning.job` object represents a fine-tuning job that has been created through the API.


The Unix timestamp (in seconds) for when the fine-tuning job was created.

For fine-tuning jobs that have `failed`, this will contain more information on the cause of the failure.

The parameter that was invalid, usually `training_file` or `validation_file`. This field will be null if the failure was not parameter-specific.

The Unix timestamp (in seconds) for when the fine-tuning job is estimated to finish. The value will be null if the fine-tuning job is not running.

The name of the fine-tuned model that is being created. The value will be null if the fine-tuning job is still running.

The Unix timestamp (in seconds) for when the fine-tuning job was finished. The value will be null if the fine-tuning job is still running.

The hyperparameters used for the fine-tuning job. See the [fine-tuning guide](https://platform.openai.com/docs/guides/fine-tuning) for more details.

The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset.
"auto" decides the optimal number of epochs based on the size of the dataset. If setting the number manually, we support any number between 1 and 50 epochs.

The object identifier, which can be referenced in the API endpoints.

The object type, which is always "fine_tuning.job".

The organization that owns the fine-tuning job.

The compiled results file ID(s) for the fine-tuning job. You can retrieve the results with the [Files API](https://platform.openai.com/docs/api-reference/files/retrieve-contents).

The current status of the fine-tuning job, which can be either `validating_files`, `queued`, `running`, `succeeded`, `failed`, or `cancelled`.

The total number of billable tokens processed by this fine-tuning job. The value will be null if the fine-tuning job is still running.

The file ID used for training. You can retrieve the training data with the [Files API](https://platform.openai.com/docs/api-reference/files/retrieve-contents).

The file ID used for validation. You can retrieve the validation results with the [Files API](https://platform.openai.com/docs/api-reference/files/retrieve-contents).

FineTuningJob

Portkey Key & Virtual Key

Portkey Key & Provider Auth & Provider Name

Portkey Key & Config

Portkey Key & Provider Auth & Provider Name & Custom Host

List your organization's fine-tuning jobs


Portkey AI is a comprehensive platform designed to streamline and enhance AI integration for developers and organizations. It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in your Generative AI apps.

What is Portkey?

Integrate Portkey and analyze your first LLM call in 2 minutes!

Make Your First Request

Portkey Features

Submit an Integration

Portkey connects with all major LLM providers and orchestration frameworks.

Overview

Learn to integrate OpenAI with Portkey, enabling seamless completions, prompt management, and advanced functionalities like streaming, function calling and fine-tuning.

OpenAI

Structured Outputs ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Portkey supports OpenAI's Structured Outputs feature out of the box with our SDKs & APIs. 

Structured Outputs

Prompt Caching

Anthropic

Google Gemini

Google Vertex AI

Controlled Generations ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Portkey supports Vertex AI's Controlled Generations feature out of the box with our SDKs & APIs. 

Controlled Generations

Azure OpenAI is a great alternative to accessing the best models including GPT-4 and more in your private environments. Portkey provides complete support for Azure OpenAI.

Azure OpenAI

AWS Bedrock

Ollama

LocalAI

Integrate vLLM-hosted custom models with Portkey and take them to production

vLLM

Integrate Trtiton-hosted custom models with Portkey and take them to production

Triton

AI21

Integrate Anyscale endpoints with Portkey seamlessly and make your OSS models production-ready

Anyscale

Cerebras

Cohere

Fireworks

Integrate Dashscope with Portkey for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning.

Dashscope

Deepinfra

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Deepbricks](https://deepbricks.ai/). 

Deepbricks

Portkey provides a robust and secure gateway to use and observe Deepgrm's Speech-to-Text API.

Deepgram

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including DeepSeek models. 

DeepSeek

Groq

Hugging Face

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on [Inference.net](https://www.inference.net/). 

Inference.net

Jina AI

Integrate Lambda with Portkey AI for seamless completions, prompt management, and advanced features like streaming and function calling.

Lambda Labs

Integrate LemonFox with Portkey for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning.

Lemonfox-AI

Lingyi (01.ai)

Mistral AI

MonsterAPIs provides access to generative AI model APIs at 80% lower costs. Connect to MonsterAPI LLM APIs seamlessly through Portkey's AI gateway.

Monster API

Moonshot

Nomic

Novita AI

OpenRouter

Perplexity AI

Predibase

Reka AI

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [SambaNova AI](https://sambanova.ai/).

SambaNova

Segmind

Stability AI

SiliconFlow

Together AI

Integrate Upstage with Portkey AI for seamless completions, prompt management, and advanced features like streaming and embedding.

Upstage AI

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including Voyage AI's embedding and Re-rank endpoints. 

Voyage AI

Workers AI

ZhipuAI / ChatGLM / BigModel

Suggest a new integration!

Bring Your Own LLM

Portkey helps bring your agents to production

Use Portkey with Autogen to take your AI Agents to production

Autogen

Use Portkey with Control Flow to take your AI Agents to production

Control Flow

Use Portkey with CrewAI to take your AI Agents to production

CrewAI

Langchain Agents

Use Portkey with LangGraph to take your AI Agents to production

LangGraph Agents

Use Portkey with Llama Agents to take your AI Agents to production

Llama Agents by Llamaindex

The Portkey x Swarm integration brings advanced AI gateway capabilities, full-stack observability, and reliability features to build production-ready AI agents.

OpenAI Swarm

Use Portkey with Phidata to take your AI Agents to production

Phidata

You can also use Portkey if you are doing custom agent orchestration!

Bring Your own Agents

AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.

Integrate DSPy with Portkey for production-ready LLM pipelines

DSPy

With Portkey, you can confidently take your Instructor pipelines to production and get complete observability over all of your calls + make them reliable - all with a 2 LOC change!

Instructor

Portkey adds core production capabilities to any Langchain app.

Langchain (Python)

Langchain (JS/TS)

Integrate Portkey with LibreChat for enhanced AI capabilities

LibreChat

Cost tracking, observability, and more for Open WebUI

Open WebUI

The **Portkey x LlamaIndex** integration brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** to apps built on LlamaIndex.

LlamaIndex (Python)

Portkey brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** + **versioning** to your **Promptfoo** projects. This document provides an overview of how to leverage the strengths of both the platforms to streamline your AI development workflow.

Promptfoo

Integrate Portkey with Vercel AI SDK for production-ready and reliable AI apps

Vercel

Integrate MindsDB with Portkey to build enterprise-grade AI use-cases

MindsDb

ToolJet is a low-code platform that lets you build apps by connecting APIs and data sources, with Portkey integration adding AI features like chat interfaces and automation.

ToolJet

MongoDB

Supabase

Gain real-time insights, track key metrics, and streamline debugging with our comprehensive observability suite.

Observability (OpenTelemetry)

The Logs section presents a chronological list of all the requests processed through Portkey.

Logs

The **Tracing** capabilities in Portkey empowers you to monitor the lifecycle of your LLM requests in a unified, chronological view.

Tracing

Analytics

Portkey's Feedback APIs provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app.

Feedback

Metadata

Filters

Easily access your Portkey logs data for further analysis and reporting

Logs Export

Budget Limits

The world's fastest AI Gateway with advanced routing & integrated Guardrails.

AI Gateway

Portkey's Universal API provides a consistent interface to integrate a wide range of modalities (text, vision, audio) and LLMs (hosted OR local) into your apps.

Universal API

This feature is available on all Portkey plans.

Configs

Conditional Routing

Cache (Simple & Semantic)

Fallbacks

LLM APIs often have inexplicable failures. With Portkey, you can rescue a substantial number of your requests with our in-built automatic retries feature. 

Automatic Retries

Load Balancing feature efficiently distributes network traffic across multiple LLMs.

Load Balancing

You can use Portkey's AI gateway to also canary test new models or prompts in different environments. 

Canary Testing

Manage unpredictable LLM latencies effectively with Portkey's **Request Timeouts**.

Request Timeouts

With Portkey's Prompt Library, you can seamlessly create and manage prompts along with all associated model parameters.

Prompt Library

With Prompt Templates, you can seamlessly create and manage your LLM prompts in one place, and deploy them with just an API call.

Prompt Templates

With Prompt Partials, you can save your commonly used templates (which could be your instruction set, data structure explanation, examples etc.) separately from your prompts and flexibly incorporate them wherever required.

Prompt Partials

See how to use Portkey's prompt templates with OpenAI (or any other provider) SDKs

Retrieve Prompts

Advanced Prompting with JSON Mode

Ship to production confidently with Portkey Guardrails on your requests & responses

API Reference

Chat Completions

Portkey Prompts

Embeddings

Images

Audio

Other APIs

Completions

Moderations

Fine-tuning

Batch

Files

Assistants

List Fine-tuning Jobs

Authorizations

Query Parameters

Response