Tracing

This is perfect for agentic workflows, chatbots, or multi-step LLM calls, by helping you understand and optimize your AI application’s performance.

​

How Tracing Works

Portkey implements OpenTelemetry-compliant tracing. When you include a trace ID with your requests, all related LLM calls are grouped together in the Traces View, appearing as “spans” within that trace.

“Span” is another word for subgrouping of LLM calls. Based on how you instrument, it can refer to another group within your trace or to a single LLM call.

​

Trace Tree Structure

Portkey uses a tree data structure for tracing, similar to OTel.

Each node in the tree is a span with a unique spanId and optional spanName. Child spans link to a parent via the parentSpanId. Parentless spans become root nodes.

traceId
├─ parentSpanId
│  ├─ spanId
│  ├─ spanName

​

Enabling Tracing

You can enable tracing by passing the trace tree values while making your request (or while instantiating your client).

Based on these values, Portkey will instrument your requests, and will show the exact trace with its spans on the “Traces” view in Logs page.

• NodeJS
• Python
• OpenAI NodeJS
• OpenAI Python
• cURL

const requestOptions = {
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
}

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-4o',
}, requestOptions);

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "VIRTUAL_KEY",
    traceId: "1729",
    spanId: "11",
    spanName: "LLM Call"
})

• NodeJS
• Python
• OpenAI NodeJS
• OpenAI Python
• cURL

const requestOptions = {traceID: "YOUR_TRACE_ID"}

const chatCompletion = await portkey.chat.completions.create({
  messages: [{ role: 'user', content: 'Say this is a test' }],
  model: 'gpt-4o',
}, requestOptions);

console.log(chatCompletion.choices);

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "VIRTUAL_KEY",
    traceID: "TRACE_ID"
})

​

See Tracing in Action

​

Tracing in Langchain

Portkey has a dedicated handler that can instrument your Langchain chains and agents to trace them.

• Trace Langchain Requests (Python)

$ pip install langchain_OpenAI portkey-ai langchain_community

from langchain_openai import ChatOpenAI
from langchain.chains import LLMChain
from portkey_ai.langchain import LangchainCallbackHandler
from portkey_ai import createHeaders

portkey_handler = LangchainCallbackHandler(
      api_key="YOUR_PORTKEY_API_KEY",
      metadata={
            "user_name": "User_Name",
            "traceId": "Langchain_sample_callback_handler"
      }
)

llm = ChatOpenAI(
    api_key="OPENAI_API_KEY",
    callbacks=[portkey_handler],
)

chain = LLMChain(
    llm=llm,
    prompt=prompt,
    callbacks=[portkey_handler]
)

handler_config = {'callbacks' : [portkey_handler]}

chain.invoke({"input": "what is langchain?"}, config=handler_config)

​

Tracing Llamaindex Requests

Portkey has a dedicated handler to instrument your Llamaindex requests on Portkey.

• Trace Llamaindex Requests

$ pip install openai portkey-ai llama-index

from llama_index.llms.openai import OpenAI
from portkey_ai.llamaindex import LlamaIndexCallbackHandler

portkey_handler = LlamaIndexCallbackHandler(
      api_key="PORTKEY_API_KEY",
      metadata={
            "user_name": "User_Name",
            "traceId": "Llamaindex_sample_callback_handler"
      }
)

llm = OpenAI(
    model="gpt-4o",
    api_key="OPENAI_API_KEY",
    callback_manager=[portkey_handler],
)

from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager

Settings.callback_manager = CallbackManager([portkey_handler])
Settings.llm = llm

​

Inserting Logs

If you are using the Insert Log API to add logs to Portkey, your traceId, spanId etc. will become part of the metadata object in your log, and Portkey will instrument your requests to take those values into account.

The logger endpoint supports inserting a single log as well as log array, and helps you build traces of any depth or complexity. For more, check here:

​

Tracing for Gateway Features

Tracing also works very well to capture the Gateway behavior on retries, fallbacks, and other routing mechanisms on Portkey Gateway.

Portkey automatically groups all the requests that were part of a single fallback or retry config and shows the failed and succeeded requests chronologically as “spans” inside a “trace”.

This is especially useful when you want to understand the total latency and behavior of your app when retry or fallbacks were triggered.

For more, check out the Fallback & Automatic Retries docs.

​

Why Use Tracing?

• Cost Insights: View aggregate LLM costs at the trace level.
• Debugging: Easily browse all requests in a single trace and identify failures.
• Performance Analysis: Understand your entire request lifecycle and total trace duration.
• User Feedback Integration: Link user feedback to specific traces for targeted improvements.

​

Capturing User Feedback

Trace IDs can also be used to link user feedback to specific generations. This can be used in a system where users provide feedback, like a thumbs up or thumbs down, or something more complex via our feedback APIs. This feedback can be linked to traces which can span over a single generation or multiple ones. Read more here: