This feature is available for all plans:-
  • Developer: 10k Logs / Month with 3 day Log Retention
  • Production: 100k Logs / Month + $9 for additional 100k with 30 Days Log Retention
  • Enterprise: Unlimited
This is perfect for agentic workflows, chatbots, or multi-step LLM calls, by helping you understand and optimize your AI application’s performance.

​
How Tracing Works

Portkey implements OpenTelemetry-compliant tracing. When you include a trace ID with your requests, all related LLM calls are grouped together in the Traces View, appearing as “spans” within that trace.
“Span” is another word for subgrouping of LLM calls. Based on how you instrument, it can refer to another group within your trace or to a single LLM call.

​
Trace Tree Structure

Portkey uses a tree data structure for tracing, similar to OTel. Each node in the tree is a span with a unique spanId and optional spanName. Child spans link to a parent via the parentSpanId. Parentless spans become root nodes. 
traceId
├─ parentSpanId
│  ├─ spanId
│  ├─ spanName
Key - NodeKey - PythonExpected ValueRequired?
traceIdtrace_idUnique stringYES
spanIdspan_idUnique stringNO
spanNamespan_namestringNO
parentSpanIdparent_span_idUnique stringNO

​
Enabling Tracing

You can enable tracing by passing the trace tree values while making your request (or while instantiating your client). Based on these values, Portkey will instrument your requests, and will show the exact trace with its spans on the “Traces” view in Logs page.
Add tracing details to a single request (recommended)
const requestOptions = {
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
}

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-4o',
}, requestOptions);

​
Or, add trace details while instantiating your client

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "VIRTUAL_KEY",
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
})
If you are only passing trace ID and not the span details, you can set the trace ID while making your request or while instantiating your client.
const requestOptions = {traceID: "YOUR_TRACE_ID"}

const chatCompletion = await portkey.chat.completions.create({
  messages: [{ role: 'user', content: 'Say this is a test' }],
  model: 'gpt-4o',
}, requestOptions);

console.log(chatCompletion.choices);

​
Pass Trace ID while instantiating your client

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "VIRTUAL_KEY",
    traceID: "TRACE_ID"
})

​
See Tracing in Action

​
Tracing in Langchain

Portkey has a dedicated handler that can instrument your Langchain chains and agents to trace them.
  1. First, install Portkey SDK, and Langchain’s packages
$ pip install langchain_OpenAI portkey-ai langchain_community
  1. Import the packages
from langchain_openai import ChatOpenAI
from langchain.chains import LLMChain
from portkey_ai.langchain import LangchainCallbackHandler
from portkey_ai import createHeaders
  1. Instantiate Portkey’s Langchain Callback Handler
portkey_handler = LangchainCallbackHandler(
      api_key="YOUR_PORTKEY_API_KEY",
      metadata={
            "user_name": "User_Name",
            "traceId": "Langchain_sample_callback_handler"
      }
)
  1. Add the callback to the ChatOpenAI instance
llm = ChatOpenAI(
    api_key="OPENAI_API_KEY",
    callbacks=[portkey_handler],
)
  1. Also add the callback when you define or run your LLM chain
chain = LLMChain(
    llm=llm,
    prompt=prompt,
    callbacks=[portkey_handler]
)

handler_config = {'callbacks' : [portkey_handler]}

chain.invoke({"input": "what is langchain?"}, config=handler_config)

​
Tracing Llamaindex Requests

Portkey has a dedicated handler to instrument your Llamaindex requests on Portkey.
  1. First, install Portkey SDK, and LlamaIndex packages
$ pip install openai portkey-ai llama-index
  1. Import the packages
from llama_index.llms.openai import OpenAI
from portkey_ai.llamaindex import LlamaIndexCallbackHandler
  1. Instantiate Portkey’s LlamaIndex Callback Handler
portkey_handler = LlamaIndexCallbackHandler(
      api_key="PORTKEY_API_KEY",
      metadata={
            "user_name": "User_Name",
            "traceId": "Llamaindex_sample_callback_handler"
      }
)
  1. Add it to OpenAI llm class
llm = OpenAI(
    model="gpt-4o",
    api_key="OPENAI_API_KEY",
    callback_manager=[portkey_handler],
)
  1. In Llama Index, you can also set the callback at a global level
from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager

Settings.callback_manager = CallbackManager([portkey_handler])
Settings.llm = llm

​
Inserting Logs

If you are using the Insert Log API to add logs to Portkey, your traceId, spanId etc. will become part of the metadata object in your log, and Portkey will instrument your requests to take those values into account. The logger endpoint supports inserting a single log as well as log array, and helps you build traces of any depth or complexity. For more, check here:

Insert a Log

​
Tracing for Gateway Features

Tracing also works very well to capture the Gateway behavior on retries, fallbacks, and other routing mechanisms on Portkey Gateway. Portkey automatically groups all the requests that were part of a single fallback or retry config and shows the failed and succeeded requests chronologically as “spans” inside a “trace”. This is especially useful when you want to understand the total latency and behavior of your app when retry or fallbacks were triggered. For more, check out the Fallback & Automatic Retries docs.

​
Why Use Tracing?

  • Cost Insights: View aggregate LLM costs at the trace level.
  • Debugging: Easily browse all requests in a single trace and identify failures.
  • Performance Analysis: Understand your entire request lifecycle and total trace duration.
  • User Feedback Integration: Link user feedback to specific traces for targeted improvements.

​
Capturing User Feedback

Trace IDs can also be used to link user feedback to specific generations. This can be used in a system where users provide feedback, like a thumbs up or thumbs down, or something more complex via our feedback APIs. This feedback can be linked to traces which can span over a single generation or multiple ones. Read more here:

Feedback