HoneyHive

HoneyHive is a comprehensive AI observability platform that helps you monitor, evaluate, and improve your LLM applications. When combined with the AI Gateway, you get powerful observability features alongside the AI Gateway’s advanced AI gateway capabilities. This integration allows you to:

Automatically trace all LLM requests through the AI Gateway
Use the AI Gateway’s 3,000+ LLMs with HoneyHive observability
Implement advanced features like caching, fallbacks, and load balancing
Maintain detailed traces and analytics in both platforms

Quick Start Integration

HoneyHive automatically traces requests to popular LLM providers, making the integration with the AI Gateway seamless. Simply initialize HoneyHive and point your LLM clients to the AI Gateway.

Installation

pip install honeyhive openai

Basic Setup

from openai import OpenAI
from honeyhive import HoneyHiveTracer, trace

# Initialize HoneyHive tracer at the beginning of your application
HoneyHiveTracer.init(
    api_key='YOUR_HONEYHIVE_API_KEY',
    project='your-project-name',
    source='production',  # Optional: dev, staging, production
    session_name='Portkey Integration'  # Optional
)

# Create OpenAI client pointing to the AI Gateway
client = OpenAI(
    api_key="YOUR_PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1"
)

# Make requests - automatically traced by HoneyHive
response = client.chat.completions.create(
    model="@openai-provider-slug/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello, world!"}]
)

print(response.choices[0].message.content)

HoneyHive automatically traces all requests to popular LLM providers, so you get observability data without additional configuration.

Using AI Gateway Features with HoneyHive

1. Trace Functions with @trace Decorator

Use HoneyHive’s @trace decorator to monitor specific functions:

@trace
def call_openai():
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )

    completion = client.chat.completions.create(
        model="@openai-provider-slug/gpt-4o-mini",
        messages=[{"role": "user", "content": "What is the meaning of life?"}]
    )

    return completion.choices[0].message.content

# Call the traced function
result = call_openai()

2. Multiple Providers

Switch between 3,000+ LLMs while maintaining HoneyHive observability:

OpenAI
Anthropic

@trace
def call_openai():
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )
    return client.chat.completions.create(
        model="@openai-provider-slug/gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello!"}]
    )

@trace
def call_anthropic():
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )
    return client.chat.completions.create(
        model="@anthropic-provider-slug/claude-3-sonnet-20240229",
        messages=[{"role": "user", "content": "Hello!"}]
    )

3. Advanced Routing with Configs

Use the AI Gateway’s config system for advanced features while tracking in HoneyHive:

@trace
def advanced_llm_call():
    # Reference a saved config from Strata Cloud Manager
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )

    return client.chat.completions.create(
        model="@config-slug/model-name",  # Your saved config ID
        messages=[{"role": "user", "content": "Analyze this data..."}]
    )

Example config for fallback between providers:

{
  "strategy": {
    "mode": "fallback"
  },
  "targets": [
    {
      "provider": "openai",
      "api_key": "YOUR_OPENAI_KEY",
      "override_params": {"model": "gpt-4o"}
    },
    {
      "provider": "anthropic",
      "api_key": "YOUR_ANTHROPIC_KEY",
      "override_params": {"model": "claude-3-opus-20240229"}
    }
  ]
}

4. Caching for Cost Optimization

Enable caching to reduce costs while maintaining full observability:

@trace
def cached_llm_call():
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )

    return client.chat.completions.create(
        model="@cache-config-slug/gpt-4o-mini",
        messages=[{"role": "user", "content": "What is machine learning?"}]
    )

5. Custom Metadata and Tracing

Add custom metadata visible in both HoneyHive and the AI Gateway:

@trace
def contextualized_llm_call(user_id, query):
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )

    return client.chat.completions.create(
        model="@openai-provider-slug/gpt-4o-mini",
        messages=[{"role": "user", "content": query}],
        # Metadata can be passed directly in the request
        user=user_id
    )

Fallbacks

Automatically switch to backup targets if the primary target fails.

Conditional Routing

Route requests to different targets based on specified conditions.

Load Balancing

Distribute requests across multiple targets based on defined weights.

Caching

Enable caching of responses to improve performance and reduce costs.

Smart Retries

Automatic retry handling with exponential backoff for failed requests

Budget Limits

Set and manage budget limits across teams and departments. Control costs with granular budget limits and usage tracking.

Observability Features

With this integration, you get:

In HoneyHive:

Automatic request/response tracing
Function-level performance metrics
Session-based analytics
Custom event tracking
Error monitoring and debugging

In the AI Gateway:

Request logs with provider details
Advanced analytics across providers
Cost tracking and budgets
Performance metrics
Custom dashboards
Token usage analytics

Migration Guide

If you’re already using HoneyHive with OpenAI, migrating to use the AI Gateway is simple:

from openai import OpenAI
from honeyhive import HoneyHiveTracer, trace

HoneyHiveTracer.init(
    api_key='YOUR_HONEYHIVE_API_KEY',
    project='my-project'
)

@trace
def call_openai():
    client = OpenAI(api_key="YOUR_OPENAI_KEY")
    return client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello!"}]
    )

from openai import OpenAI
from honeyhive import HoneyHiveTracer, trace

HoneyHiveTracer.init(
    api_key='YOUR_HONEYHIVE_API_KEY',
    project='my-project'
)

@trace
def call_openai():
    client = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1"
    )
    return client.chat.completions.create(
        model="@openai-provider-slug/gpt-4",
        messages=[{"role": "user", "content": "Hello!"}]
    )

Resources

For enterprise support and custom features, contact our [enterprise team](https://www.paloaltonetworks.com/ai-security/ai-gateway#:~:text=See%20Prisma%20AIRS%20AI%20Gateway%20in%20Action

Integrations

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

MCP Clients

MCP Servers

Quick Start Integration

Installation

Basic Setup

Using AI Gateway Features with HoneyHive

1. Trace Functions with @trace Decorator

2. Multiple Providers

3. Advanced Routing with Configs

4. Caching for Cost Optimization

5. Custom Metadata and Tracing

Fallbacks

Conditional Routing

Load Balancing

Caching

Smart Retries

Budget Limits

Observability Features

In HoneyHive:

In the AI Gateway:

Migration Guide

Resources

​Quick Start Integration

​Installation

​Basic Setup

​Using AI Gateway Features with HoneyHive

​1. Trace Functions with @trace Decorator

​2. Multiple Providers

​3. Advanced Routing with Configs

​4. Caching for Cost Optimization

​5. Custom Metadata and Tracing

Fallbacks

Conditional Routing

Load Balancing

Caching

Smart Retries

Budget Limits

​Observability Features

​In HoneyHive:

​In the AI Gateway:

​Migration Guide

​Resources

Quick Start Integration

Installation

Basic Setup

Using AI Gateway Features with HoneyHive

1. Trace Functions with @trace Decorator

2. Multiple Providers

3. Advanced Routing with Configs

4. Caching for Cost Optimization

5. Custom Metadata and Tracing

Observability Features

In HoneyHive:

In the AI Gateway:

Migration Guide

Resources