OpenTelemetry Python SDK

The OpenTelemetry SDK provides direct, fine-grained control over instrumentation in your LLM applications. Unlike automatic instrumentation libraries, the SDK allows you to manually create spans and set attributes exactly where and how you need them.

Using the OpenTelemetry SDK directly with Prisma AIRS AI Gateway gives you complete control over what gets traced while benefiting from the AI Gateway’s intelligent gateway features like caching, fallbacks, and load balancing.

Why OpenTelemetry SDK + the AI Gateway?

Full Control

Manually instrument exactly what you need with custom spans and attributes

Production Ready

Battle-tested OpenTelemetry standard used by enterprises worldwide

Custom Attributes

Add any metadata you need to traces for debugging and analysis

Gateway Intelligence

The AI Gateway adds routing optimization and resilience to your LLM calls

Quick Start

Prerequisites

Python
Strata Cloud Manager account with API key
OpenAI API key (or add it to Model Catalog)

Step 1: Install Dependencies

Install the required packages:

pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http openai

Step 2: Configure OpenTelemetry

Set up the tracer provider and OTLP exporter:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

# Setup tracer provider
provider = TracerProvider()
trace.set_tracer_provider(provider)

# Configure OTLP exporter to send to the AI Gateway
otlp_exporter = OTLPSpanExporter(
    endpoint="https://aigw.portkey.ai/v1/otel/v1/traces",
    headers={
        "Authorization": "Bearer YOUR_PORTKEY_API_KEY",
    }
)

# Add batch span processor (recommended for production)
span_processor = BatchSpanProcessor(otlp_exporter)
provider.add_span_processor(span_processor)

# Get tracer
tracer = trace.get_tracer(__name__)

Step 3: Configure AI Gateway

Set up the OpenAI client with the AI Gateway:

from openai import OpenAI

# Use the AI Gateway for intelligent routing
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={
        "x-portkey-provider": "@openai-prod",  # Your AI Provider slug from Model Catalog
    }
)

Step 4: Create Instrumented Functions

Manually instrument your LLM calls with custom spans:

def generate_ai_response(input_text):
    with tracer.start_as_current_span("OpenAI-Chat-Completion") as span:
        # Add input attributes
        span.set_attribute("input.value", input_text)
        span.set_attribute("model.name", "gpt-4o")
        span.set_attribute("temperature", 0.7)
        span.set_attribute("gen_ai.prompt.0.role", "user")
        span.set_attribute("gen_ai.prompt.0.content", input_text)

        # Make the API call
        response = client.chat.completions.create(
            messages=[{"role": "user", "content": input_text}],
            model="gpt-4o",
            temperature=0.7,
        )

        # Add response attributes
        output_content = response.choices[0].message.content
        span.set_attribute("output.value", output_content)
        span.set_attribute("gen_ai.completion.0.role", "assistant")
        span.set_attribute("gen_ai.completion.0.content", output_content)

        return output_content

Complete Example

Here’s a full working example:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openai import OpenAI

# Step 1: Setup OpenTelemetry
provider = TracerProvider()
trace.set_tracer_provider(provider)

otlp_exporter = OTLPSpanExporter(
    endpoint="https://aigw.portkey.ai/v1/otel/v1/traces",
    headers={"Authorization": "Bearer YOUR_PORTKEY_API_KEY"}
)

span_processor = BatchSpanProcessor(otlp_exporter)
provider.add_span_processor(span_processor)

tracer = trace.get_tracer(__name__)

# Step 2: Configure the AI Gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1"
)

# Step 3: Create instrumented function
def generate_ai_response(input_text):
    with tracer.start_as_current_span("OpenAI-Chat-Completion") as span:
        # Set input attributes
        span.set_attribute("input.value", input_text)
        span.set_attribute("model.name", "gpt-4o")
        span.set_attribute("temperature", 0.7)

        # Make API call
        response = client.chat.completions.create(
            messages=[{"role": "user", "content": input_text}],
            model="@openai-prod/gpt-4o",
            temperature=0.7,
        )

        # Set output attributes
        output_content = response.choices[0].message.content
        span.set_attribute("output.value", output_content)

        return output_content

# Step 4: Use the instrumented function
if __name__ == "__main__":
    input_text = "Explain the concept of AI in 50 words"
    response = generate_ai_response(input_text)
    print(response)

Next Steps

Configure Gateway

Set up intelligent routing, fallbacks, and caching

Model Catalog

Manage AI providers, credentials, and model access centrally

View Analytics

Analyze costs, performance, and usage patterns

Set Up Alerts

Configure alerts for anomalies and performance issues

See Your Traces in Action

Once configured, navigate to the Strata Cloud Manager to see your custom OpenTelemetry traces enhanced with gateway intelligence:

Integrations

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

MCP Clients

MCP Servers

OpenTelemetry Python SDK

Why OpenTelemetry SDK + the AI Gateway?

Full Control

Production Ready

Custom Attributes

Gateway Intelligence

Quick Start

Prerequisites

Step 1: Install Dependencies

Step 2: Configure OpenTelemetry

Step 3: Configure AI Gateway

Step 4: Create Instrumented Functions

Complete Example

Next Steps

Configure Gateway

Model Catalog

View Analytics

Set Up Alerts

See Your Traces in Action

​Why OpenTelemetry SDK + the AI Gateway?

Full Control

Production Ready

Custom Attributes

Gateway Intelligence

​Quick Start

​Prerequisites

​Step 1: Install Dependencies

​Step 2: Configure OpenTelemetry

​Step 3: Configure AI Gateway

​Step 4: Create Instrumented Functions

​Complete Example

​Next Steps

Configure Gateway

Model Catalog

View Analytics

Set Up Alerts

​See Your Traces in Action

Why OpenTelemetry SDK + the AI Gateway?

Quick Start

Prerequisites

Step 1: Install Dependencies

Step 2: Configure OpenTelemetry

Step 3: Configure AI Gateway

Step 4: Create Instrumented Functions

Complete Example

Next Steps

See Your Traces in Action