OpenLIT

OpenLIT allows you to simplify your AI development workflow, especially for Generative AI and LLMs. It streamlines essential tasks like experimenting with LLMs, organizing and versioning prompts, and securely handling API keys. With just one line of code, you can enable OpenTelemetry-native observability, offering full-stack monitoring that includes LLMs, vector databases, and GPUs.

OpenLIT’s automatic instrumentation combined with Prisma AIRS AI Gateway’s intelligent gateway creates a comprehensive observability solution where every trace captures model performance, prompt versioning, and cost optimization data in real-time.

Why OpenLIT + the AI Gateway?

One-Line Instrumentation

Enable complete observability with a single line of code for all AI components

Full-Stack AI Monitoring

Monitor LLMs, vector databases, and GPUs in a unified view

Native OpenTelemetry

Built on OpenTelemetry standards for seamless integration

Production-Ready

Smooth transition from experimentation to production deployment

Quick Start

Prerequisites

Python
Strata Cloud Manager account with API key
OpenAI API key (or add it to Model Catalog)

Step 1: Install Dependencies

Install the required packages for OpenLIT and AI Gateway integration:

pip install openlit openai

Step 2: Configure OpenTelemetry Export

Set up the environment variables to send traces to the AI Gateway’s OpenTelemetry endpoint:

import os

# Configure the AI Gateway endpoint and authentication
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://aigw.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "Authorization=Bearer YOUR_PORTKEY_API_KEY"

Step 3: Initialize OpenLIT with Custom Tracer

Set up OpenTelemetry tracer and initialize OpenLIT:

import openlit
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry import trace

# Create and configure the tracer provider
trace_provider = TracerProvider()
trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter()))

# Set the global default tracer provider
trace.set_tracer_provider(trace_provider)

# Create a tracer from the global tracer provider
tracer = trace.get_tracer(__name__)

# Initialize OpenLIT with the custom tracer
# disable_batch=True ensures traces are processed immediately
openlit.init(tracer=tracer, disable_batch=True)

Step 4: Configure AI Gateway

Set up the OpenAI client to use the AI Gateway’s intelligent gateway:

from openai import OpenAI

# Create OpenAI client with the AI Gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={
        "x-portkey-provider": "@openai-prod",  # Your AI Provider slug from Model Catalog
    }
)

Step 5: Make Instrumented LLM Calls

Now your LLM calls are automatically traced by OpenLIT and enhanced by the AI Gateway:

# Make calls through the AI Gateway
# OpenLIT instruments the call, the AI Gateway adds routing intelligence
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain the benefits of OpenTelemetry in AI applications"}],
    temperature=0.7
)

print(response.choices[0].message.content)

# You now get:
# 1. Automatic tracing from OpenLIT
# 2. Gateway features (caching, fallbacks, routing)
# 3. Combined insights in Strata Cloud Manager

Complete Example

Here’s a full example bringing everything together:

import os
import openlit
from openai import OpenAI
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry import trace

# Step 1: Configure the AI Gateway endpoint
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://aigw.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "Authorization=Bearer YOUR_PORTKEY_API_KEY"

# Step 2: Set up OpenTelemetry
trace_provider = TracerProvider()
trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(trace_provider)
tracer = trace.get_tracer(__name__)

# Step 3: Initialize OpenLIT
openlit.init(tracer=tracer, disable_batch=True)

# Step 4: Configure the AI Gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1"
)

# Step 5: Make instrumented calls
response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What are the key benefits of observability in AI?"}
    ]
)

print(response.choices[0].message.content)

Next Steps

Configure Gateway

Set up intelligent routing, fallbacks, and caching

Model Catalog

Manage AI providers, credentials, and model access centrally

View Analytics

Analyze costs, performance, and usage patterns

Set Up Alerts

Configure alerts for anomalies and performance issues

See Your Traces in Action

Once configured, navigate to the Strata Cloud Manager to see your OpenLIT instrumentation combined with gateway intelligence:

Integrations

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

MCP Clients

MCP Servers

Why OpenLIT + the AI Gateway?

One-Line Instrumentation

Full-Stack AI Monitoring

Native OpenTelemetry

Production-Ready

Quick Start

Prerequisites

Step 1: Install Dependencies

Step 2: Configure OpenTelemetry Export

Step 3: Initialize OpenLIT with Custom Tracer

Step 4: Configure AI Gateway

Step 5: Make Instrumented LLM Calls

Complete Example

Next Steps

Configure Gateway

Model Catalog

View Analytics

Set Up Alerts

See Your Traces in Action

​Why OpenLIT + the AI Gateway?

One-Line Instrumentation

Full-Stack AI Monitoring

Native OpenTelemetry

Production-Ready

​Quick Start

​Prerequisites

​Step 1: Install Dependencies

​Step 2: Configure OpenTelemetry Export

​Step 3: Initialize OpenLIT with Custom Tracer

​Step 4: Configure AI Gateway

​Step 5: Make Instrumented LLM Calls

​Complete Example

​Next Steps

Configure Gateway

Model Catalog

View Analytics

Set Up Alerts

​See Your Traces in Action

Why OpenLIT + the AI Gateway?

Quick Start

Prerequisites

Step 1: Install Dependencies

Step 2: Configure OpenTelemetry Export

Step 3: Initialize OpenLIT with Custom Tracer

Step 4: Configure AI Gateway

Step 5: Make Instrumented LLM Calls

Complete Example

Next Steps

See Your Traces in Action