Langsmith

• Track all LLM requests in LangSmith while routing through the AI Gateway
• Use the AI Gateway’s 3,000+ LLMs with LangSmith observability
• Implement advanced features like caching, fallbacks, and load balancing
• Maintain detailed traces and analytics in both platforms

​

Quick Start Integration

​

Installation

pip install langsmith

​

Basic Setup

​

Using AI Gateway Features with LangSmith

​

1. LLM Integrations

​

2. Multiple Providers

client = wrap_openai(Portkey(api_key="YOUR_PORTKEY_API_KEY"))

response = client.chat.completions.create(
    model="@openai-integration/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

client = wrap_openai(Portkey(api_key="YOUR_PORTKEY_API_KEY"))

response = client.chat.completions.create(
    model="@anthropic-integration/claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello!"}]
)

client = wrap_openai(Portkey(api_key="YOUR_PORTKEY_API_KEY"))

response = client.chat.completions.create(
    model="@azure-openai-integration/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

​

3. Advanced Routing with Configs

{
  "strategy": {
    "mode": "fallback"
  },
  "targets": [
    {
      "integration": "@openai-integration",
      "override_params": {"model": "gpt-4o"}
    },
    {
      "integration": "@anthropic-integration",
      "override_params": {"model": "claude-3-opus-20240229"}
    }
  ]
}

​

4. Caching for Cost Optimization

​

5. Custom Metadata and Tracing

​

Observability Features

​

In LangSmith:

• Request/response logging
• Latency tracking
• Token usage analytics
• Cost calculation
• Trace visualization

​

In the AI Gateway:

• Request logs with provider details
• Advanced analytics across providers
• Cost tracking and budgets
• Performance metrics
• Custom dashboards
• Token usage analytics

​

Migration Guide

from openai import OpenAI
from langsmith.wrappers import wrap_openai

client = wrap_openai(OpenAI(api_key="YOUR_OPENAI_KEY"))

​

Next Steps

• Create LLM Integrations for secure API key management
• Build Configs for advanced routing
• Set up Guardrails for content filtering
• Implement Caching for cost optimization

​

Resources

• LangSmith Documentation
• AI Gateway Guide