> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Tracing

> The **Tracing** capabilities in Portkey empowers you to monitor the lifecycle of your LLM requests in a unified, chronological view.

<Info>
  This feature is available for all plans:-

  * [Developer](https://app.portkey.ai/): 10k Logs / Month with 3 day Log Retention
  * [Production](https://app.portkey.ai/): 100k Logs / Month + \$9 for additional 100k with 30 Days Log Retention
  * [Enterprise](https://portkey.ai/docs/product/enterprise-offering): Unlimited
</Info>

This is perfect for **agentic workflows**, **chatbots**, or **multi-step LLM calls**, by helping you understand and optimize your AI application's performance.

<Frame>
  <img src="https://mintcdn.com/portkey-docs/_Cb_bj7tVjxcfwsu/images/product/product-11-1.webp?fit=max&auto=format&n=_Cb_bj7tVjxcfwsu&q=85&s=d64ccfdd125ef6ca39bc3e344d7411a4" width="2224" height="1166" data-path="images/product/product-11-1.webp" />
</Frame>

## How Tracing Works

Portkey implements OpenTelemetry-compliant tracing. When you include a `trace ID` with your requests, all related LLM calls are grouped together in the Traces View, appearing as "spans" within that trace.

> "Span" is another word for subgrouping of LLM calls. Based on how you instrument, it can refer to another group within your trace or to a single LLM call.

## Trace Tree Structure

Portkey uses a tree data structure for tracing, **similar to OTel.**

Each node in the tree is a span with a unique `spanId` and optional `spanName`. Child spans link to a parent via the `parentSpanId`. Parentless spans become root nodes.

```
traceId
├─ parentSpanId
│  ├─ spanId
│  ├─ spanName
```

| Key - Node   | Key - Python     | Expected Value | Required? |
| ------------ | ---------------- | -------------- | --------- |
| traceId      | trace\_id        | Unique string  | YES       |
| spanId       | span\_id         | Unique string  | NO        |
| spanName     | span\_name       | string         | NO        |
| parentSpanId | parent\_span\_id | Unique string  | NO        |

***

## W3C Trace Context Support

Portkey supports the [W3C Trace Context](https://www.w3.org/TR/trace-context/) standard headers, making it easy to integrate with existing OpenTelemetry-instrumented applications. If you're already using OpenTelemetry in your stack, you can pass the standard `traceparent` and `baggage` headers instead of Portkey-specific headers.

### traceparent Header

The `traceparent` header follows the W3C format: `version-trace_id-parent_id-trace_flags`

**Example:** `00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00`

When you send a request with the `traceparent` header:

* The `trace_id` (32 hex characters) is extracted and used as `x-portkey-trace-id`
* The `parent_id` (16 hex characters) is extracted and used as `x-portkey-parent-span-id` (the caller's span)
* A new `span_id` is automatically generated for the gateway's span
* The `span_name` is automatically set to the HTTP method and path (e.g., `POST /v1/chat/completions`)

<Note>
  If you provide both `traceparent` and Portkey-specific headers (`x-portkey-trace-id`, `x-portkey-span-id`), the Portkey-specific headers take precedence.
</Note>

### baggage Header

The `baggage` header allows you to pass key-value metadata following the [W3C Baggage](https://www.w3.org/TR/baggage/) specification.

**Example:** `userId=alice,environment=production,requestType=chat`

When you send a request with the `baggage` header:

* Key-value pairs are parsed and merged into the request metadata
* Existing metadata (from `x-portkey-metadata` header) takes precedence over baggage values
* Values are URL-decoded automatically

### Example: Using W3C Headers

<Tabs>
  <Tab title="cURL">
    ```sh theme={"system"}
    curl https://api.portkey.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -H "x-portkey-api-key: $PORTKEY_API_KEY" \
      -H "x-portkey-provider: openai" \
      -H "traceparent: 00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00" \
      -H "baggage: userId=alice,environment=production" \
      -d '{
        "model": "gpt-4o",
        "messages": [{"role": "user","content": "Hello!"}]
      }'
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"system"}
    import requests

    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {OPENAI_API_KEY}",
        "x-portkey-api-key": PORTKEY_API_KEY,
        "x-portkey-provider": "openai",
        "traceparent": "00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00",
        "baggage": "userId=alice,environment=production"
    }

    response = requests.post(
        "https://api.portkey.ai/v1/chat/completions",
        headers=headers,
        json={
            "model": "gpt-4o",
            "messages": [{"role": "user", "content": "Hello!"}]
        }
    )
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={"system"}
    const response = await fetch('https://api.portkey.ai/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${OPENAI_API_KEY}`,
        'x-portkey-api-key': PORTKEY_API_KEY,
        'x-portkey-provider': 'openai',
        'traceparent': '00-bad90143930ea019df8f681254fb2393-42df8ac2dde4ac52-00',
        'baggage': 'userId=alice,environment=production'
      },
      body: JSON.stringify({
        model: 'gpt-4o',
        messages: [{ role: 'user', content: 'Hello!' }]
      })
    });
    ```
  </Tab>
</Tabs>

<Info>
  W3C Trace Context support is particularly useful when integrating Portkey into applications that already use OpenTelemetry for distributed tracing, as it allows seamless correlation between your existing traces and LLM calls.
</Info>

***

## Enabling Tracing

You can enable tracing by passing the `trace tree` values while making your request (or while instantiating your client).

Based on these values, Portkey will instrument your requests, and will show the exact trace with its spans on the "Traces" view in Logs page.

<Tabs>
  <Tab title="NodeJS">
    **Add tracing details to a single request (recommended)**

    ```js theme={"system"}
    const requestOptions = {
        traceId: "1729",
        spanId: "11",
        spanName: "LLM Call"
    }

    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'gpt-4o',
    }, requestOptions);
    ```

    #### Or, add trace details while instantiating your client

    ```js theme={"system"}
    import Portkey from 'portkey-ai';

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY",
        provider:"@PROVIDER",
        traceId: "1729",
        spanId: "11",
        spanName: "LLM Call"
    })
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"system"}
    completion = portkey.with_options(
        trace_id="1729",
        span_id="11",
        span_name="LLM Call"
    ).chat.completions.create(
        messages = [{ "role": 'user', "content": 'Say this is a test' }],
        model = 'gpt-3.5-turbo'
    )
    ```

    #### Pass Trace details while instantiating your client

    ```py theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",
        provider="@PROVIDER",
        trace_id="1729",
        span_id="11",
        span_name="LLM Call"
    )
    ```
  </Tab>

  <Tab title="OpenAI NodeJS">
    ```js theme={"system"}
    import { createHeaders } from 'portkey-ai'

    const requestOptions = {
        traceId: "1729",
        spanId: "11",
        spanName: "LLM Call"
    }

    const chatCompletion = await openai.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'gpt-3.5-turbo',
    }, requestOptions);
    ```
  </Tab>

  <Tab title="OpenAI Python">
    ```py theme={"system"}
    from portkey_ai import createHeaders

    req_headers = createHeaders(
        trace_id="1729",
        span_id="11",
        span_name="LLM Call
    )

    chat_complete = client.with_options(headers=req_headers).chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Say this is a test"}],
    )
    ```
  </Tab>

  <Tab title="cURL">
    ```sh theme={"system"}
    curl https://api.portkey.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -H "x-portkey-api-key: $PORTKEY_API_KEY" \
      -H "x-portkey-provider: openai" \
      -H "x-portkey-trace-id: 1729"\
      -H "x-portkey-span-id: 11"\
      -H "x-portkey-span-name: LLM_CALL"\
      -d '{
        "model": "gpt-4o",
        "messages": [{"role": "user","content": "Hello!"}]
      }'
    ```
  </Tab>
</Tabs>

<Info>
  If you are only passing trace ID and not the span details, you can set the trace ID while making your request or while instantiating your client.
</Info>

<Tabs>
  <Tab title="NodeJS">
    ```js theme={"system"}
    const requestOptions = {traceID: "YOUR_TRACE_ID"}

    const chatCompletion = await portkey.chat.completions.create({
      messages: [{ role: 'user', content: 'Say this is a test' }],
      model: 'gpt-4o',
    }, requestOptions);

    console.log(chatCompletion.choices);
    ```

    #### Pass Trace ID while instantiating your client

    ```js theme={"system"}
    import Portkey from 'portkey-ai';

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY",
        provider:"@PROVIDER",
        traceID: "TRACE_ID"
    })
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"system"}
    completion = portkey.with_options(
        trace_id = "TRACE_ID"
    ).chat.completions.create(
        messages = [{ "role": 'user', "content": 'Say this is a test' }],
        model = 'gpt-3.5-turbo'
    )
    ```

    #### Pass Trace ID while instantiating your client

    ```py theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",
        provider="@PROVIDER",
        trace_id="TRACE_ID"
    )
    ```
  </Tab>

  <Tab title="OpenAI NodeJS">
    ```js theme={"system"}
    import { createHeaders } from 'portkey-ai'

    const reqHeaders = {headers: createHeaders({"traceID": "TRACE_ID"})}

    const chatCompletion = await openai.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'gpt-3.5-turbo',
    }, reqHeaders);
    ```
  </Tab>

  <Tab title="OpenAI Python">
    ```py theme={"system"}
    from portkey_ai import createHeaders

    req_headers = createHeaders(trace_id="TRACE_ID")

    chat_complete = client.with_options(headers=req_headers).chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Say this is a test"}],
    )
    ```
  </Tab>

  <Tab title="cURL">
    ```sh theme={"system"}
    curl https://api.portkey.ai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -H "x-portkey-api-key: $PORTKEY_API_KEY" \
      -H "x-portkey-provider: openai" \
      -H "x-portkey-trace-id: TRACE_ID" \
      -d '{
        "model": "gpt-4-turbo",
        "messages": [{
            "role": "system",
            "content": "You are a helpful assistant."
          },{
            "role": "user",
            "content": "Hello!"
          }]
      }'
    ```
  </Tab>
</Tabs>

## See Tracing in Action

<iframe width="100%" height="350" src="https://www.youtube.com/embed/Y-EKz_tVNHs?si=Ycv93wxZCLInlwaf" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />

## Tracing in Langchain

Portkey has a dedicated handler that can instrument your Langchain chains and agents to trace them.

<Tabs>
  <Tab title="Trace Langchain Requests (Python)">
    1. First, install Portkey SDK, and Langchain's packages

    ```sh theme={"system"}
    $ pip install langchain_OpenAI portkey-ai langchain_community
    ```

    2. Import the packages

    ```py theme={"system"}
    from langchain_openai import ChatOpenAI
    from langchain.chains import LLMChain
    from portkey_ai.langchain import LangchainCallbackHandler
    from portkey_ai import createHeaders
    ```

    3. Instantiate Portkey's Langchain Callback Handler

    ```py theme={"system"}
    portkey_handler = LangchainCallbackHandler(
          api_key="YOUR_PORTKEY_API_KEY",
          metadata={
                "user_name": "User_Name",
                "traceId": "Langchain_sample_callback_handler"
          }
    )
    ```

    4. Add the callback to the `ChatOpenAI` instance

    ```py theme={"system"}
    llm = ChatOpenAI(
        api_key="OPENAI_API_KEY",
        callbacks=[portkey_handler],
    )
    ```

    5. Also add the callback when you define or run your LLM chain

    ```py theme={"system"}
    chain = LLMChain(
        llm=llm,
        prompt=prompt,
        callbacks=[portkey_handler]
    )

    handler_config = {'callbacks' : [portkey_handler]}

    chain.invoke({"input": "what is langchain?"}, config=handler_config)
    ```
  </Tab>
</Tabs>

***

## Tracing Llamaindex Requests

Portkey has a dedicated handler to instrument your Llamaindex requests on Portkey.

<Tabs>
  <Tab title="Trace Llamaindex Requests">
    1. First, install Portkey SDK, and LlamaIndex packages

    ```sh theme={"system"}
    $ pip install openai portkey-ai llama-index
    ```

    2. Import the packages

    ```py theme={"system"}
    from llama_index.llms.openai import OpenAI
    from portkey_ai.llamaindex import LlamaIndexCallbackHandler
    ```

    3. Instantiate Portkey's LlamaIndex Callback Handler

    ```py theme={"system"}
    portkey_handler = LlamaIndexCallbackHandler(
          api_key="PORTKEY_API_KEY",
          metadata={
                "user_name": "User_Name",
                "traceId": "Llamaindex_sample_callback_handler"
          }
    )
    ```

    4. Add it to `OpenAI` llm class

    ```py theme={"system"}
    llm = OpenAI(
        model="gpt-4o",
        api_key="OPENAI_API_KEY",
        callback_manager=[portkey_handler],
    )
    ```

    5. In Llama Index, you can also set the callback at a global level

    ```python theme={"system"}
    from llama_index.core import Settings
    from llama_index.core.callbacks import CallbackManager

    Settings.callback_manager = CallbackManager([portkey_handler])
    Settings.llm = llm
    ```
  </Tab>
</Tabs>

***

## Inserting Logs

If you are using the [Insert Log API](/portkey-endpoints/logs/insert-a-log) to add logs to Portkey, your `traceId`, `spanId` etc. will become part of the metadata object in your log, and Portkey will instrument your requests to take those values into account.

The logger endpoint supports inserting a single log as well as log array, and helps you build traces of any depth or complexity. For more, check here:

<Card title="Insert a Log" href="/portkey-endpoints/logs/insert-a-log" />

***

## Tracing for Gateway Features

Tracing also works very well to capture the Gateway behavior on retries, fallbacks, and other routing mechanisms on Portkey Gateway.

Portkey automatically groups all the requests that were part of a single fallback or retry config and shows the failed and succeeded requests chronologically as "spans" inside a "trace".

This is especially useful when you want to understand the total latency and behavior of your app when retry or fallbacks were triggered.

For more, check out the [Fallback](/product/ai-gateway/fallbacks) & [Automatic Retries](/product/ai-gateway/automatic-retries) docs.

***

## Why Use Tracing?

* **Cost Insights**: View aggregate LLM costs at the trace level.
* **Debugging**: Easily browse all requests in a single trace and identify failures.
* **Performance Analysis**: Understand your entire request lifecycle and total trace duration.
* **User Feedback Integration**: Link user feedback to specific traces for targeted improvements.

***

## Capturing User Feedback

Trace IDs can also be used to link user feedback to specific generations. This can be used in a system where users provide feedback, like a thumbs up or thumbs down, or something more complex via our feedback APIs. This feedback can be linked to traces which can span over a single generation or multiple ones. Read more here:

<Card title="Feedback" href="/product/observability/feedback" />