Arize Phoenix

Arize Phoenix is the open-source AI observability and evaluation platform from Arize AI. Use it for local development, OSS workflows, and self-hosted tracing with OpenInference instrumentation. For the full-featured production platform built for AI-native teams and enterprises, use Arize AX with the AI Gateway’s Arize AX integration. AX is available as managed cloud or enterprise self-hosted deployment. Arize’s agent evaluation guide and LLM evaluation guide show how traces support debugging, scoring, and continuous improvement workflows.

Phoenix’s OpenInference instrumentation combined with the AI Gateway’s intelligent gateway provides comprehensive debugging capabilities with automatic trace collection, while adding routing optimization and resilience features to your LLM calls.

Why Arize Phoenix + the AI Gateway?

Visual Debugging

Powerful UI for exploring traces, spans, and debugging LLM behavior

OpenInference Standard

Industry-standard semantic conventions for AI/LLM observability

Evaluation Tools

Built-in tools for evaluating model performance and behavior

Gateway Intelligence

The AI Gateway adds caching, fallbacks, and load balancing to every request

Quick Start

Prerequisites

Python
Strata Cloud Manager account with API key
OpenAI API key (or add it to Model Catalog)

Step 1: Install Dependencies

Install the required packages for Phoenix and AI Gateway integration:

pip install arize-phoenix-otel openai openinference-instrumentation-openai

Step 2: Configure OpenTelemetry Export

Set up the environment variables to send traces to the AI Gateway:

import os

# Configure the AI Gateway endpoint and authentication
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://aigw.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "Authorization=Bearer YOUR_PORTKEY_API_KEY"

Step 3: Register Phoenix and Instrument OpenAI

Initialize Phoenix and enable OpenAI instrumentation:

from phoenix.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor

# Configure Phoenix tracer
register(set_global_tracer_provider=False)

# Instrument OpenAI
OpenAIInstrumentor().instrument()

Step 4: Configure AI Gateway

Set up the OpenAI client with the AI Gateway:

from openai import OpenAI

# Use the AI Gateway for intelligent routing
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={
        "x-portkey-provider": "@openai-prod",  # Your AI Provider slug from Model Catalog
    }
)

Step 5: Make Instrumented LLM Calls

Your LLM calls are now automatically traced by Phoenix and enhanced by the AI Gateway:

# Make calls with automatic tracing
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "How does Phoenix help with AI debugging?"}],
    model="gpt-4o",
    temperature=0.7
)

print(response.choices[0].message.content)

# Phoenix captures:
# - Input/output pairs
# - Token usage
# - Latency metrics
# - Model parameters
#
# the AI Gateway adds:
# - Gateway routing decisions
# - Cache hit/miss data
# - Fallback information

Complete Example

Here’s a full working example:

from phoenix.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor
import os
from openai import OpenAI

# Step 1: Configure the AI Gateway endpoint
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://aigw.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "Authorization=Bearer YOUR_PORTKEY_API_KEY"

# Step 2: Register Phoenix and instrument OpenAI
register(set_global_tracer_provider=False)
OpenAIInstrumentor().instrument()

# Step 3: Configure the AI Gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1"
)

# Step 4: Make instrumented calls
response = client.chat.completions.create(
    messages=[
        {"role": "system", "content": "You are a helpful AI assistant."},
        {"role": "user", "content": "Explain how observability helps in production AI systems"}
    ],
    model="@openai-prod/gpt-4o",
    temperature=0.7
)

print(response.choices[0].message.content)

OpenInference Instrumentation

Phoenix uses OpenInference semantic conventions for AI observability:

Automatic Capture

Messages: Full conversation history with roles and content
Model Info: Model name, temperature, and other parameters
Token Usage: Input/output token counts for cost tracking
Errors: Detailed error information when requests fail
Latency: End-to-end request timing

Supported Providers

Phoenix can instrument multiple LLM providers:

OpenAI
Anthropic
Bedrock
Vertex AI
Azure OpenAI
And more through OpenInference instrumentors

Configuration Options

Custom Span Attributes

Add custom attributes to your traces:

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("custom_operation") as span:
    span.set_attribute("user.id", "user123")
    span.set_attribute("session.id", "session456")

    # Your LLM call here
    response = client.chat.completions.create(...)

Sampling Configuration

Control trace sampling for production environments:

from opentelemetry.sdk.trace.sampling import TraceIdRatioBased

# Sample 10% of traces
register(
    set_global_tracer_provider=False,
    sampler=TraceIdRatioBased(0.1)
)

Troubleshooting

Common Issues

Traces not appearing in the AI Gateway

Ensure both OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS are correctly set

Missing instrumentation data

Make sure to call OpenAIInstrumentor().instrument() before creating your OpenAI client

Phoenix UI not showing traces

If using Phoenix UI locally, ensure Phoenix is running and properly configured

Next Steps

Configure Gateway

Set up intelligent routing, fallbacks, and caching

Model Catalog

Manage AI providers, credentials, and model access centrally

View Analytics

Analyze costs, performance, and usage patterns

Set Up Evaluations

Create custom evaluations for your AI system

See Your Traces in Action

Once configured, navigate to the Strata Cloud Manager to see your Phoenix instrumentation combined with gateway intelligence:

Integrations

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

MCP Clients

MCP Servers

Why Arize Phoenix + the AI Gateway?

Visual Debugging

OpenInference Standard

Evaluation Tools

Gateway Intelligence

Quick Start

Prerequisites

Step 1: Install Dependencies

Step 2: Configure OpenTelemetry Export

Step 3: Register Phoenix and Instrument OpenAI

Step 4: Configure AI Gateway

Step 5: Make Instrumented LLM Calls

Complete Example

OpenInference Instrumentation

Automatic Capture

Supported Providers

Configuration Options

Custom Span Attributes

Sampling Configuration

Troubleshooting

Common Issues

Next Steps

Configure Gateway

Model Catalog

View Analytics

Set Up Evaluations

See Your Traces in Action

​Why Arize Phoenix + the AI Gateway?

Visual Debugging

OpenInference Standard

Evaluation Tools

Gateway Intelligence

​Quick Start

​Prerequisites

​Step 1: Install Dependencies

​Step 2: Configure OpenTelemetry Export

​Step 3: Register Phoenix and Instrument OpenAI

​Step 4: Configure AI Gateway

​Step 5: Make Instrumented LLM Calls

​Complete Example

​OpenInference Instrumentation

​Automatic Capture

​Supported Providers

​Configuration Options

​Custom Span Attributes

​Sampling Configuration

​Troubleshooting

​Common Issues

​Next Steps

Configure Gateway

Model Catalog

View Analytics

Set Up Evaluations

​See Your Traces in Action

Why Arize Phoenix + the AI Gateway?

Quick Start

Prerequisites

Step 1: Install Dependencies

Step 2: Configure OpenTelemetry Export

Step 3: Register Phoenix and Instrument OpenAI

Step 4: Configure AI Gateway

Step 5: Make Instrumented LLM Calls

Complete Example

OpenInference Instrumentation

Automatic Capture

Supported Providers

Configuration Options

Custom Span Attributes

Sampling Configuration

Troubleshooting

Common Issues

Next Steps

See Your Traces in Action