> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Langsmith

> Integrate LangSmith observability with Prisma AIRS AI Gateway's AI gateway for comprehensive LLM monitoring and advanced routing capabilities

LangSmith is LangChain's observability platform that helps you debug, test, evaluate, and monitor your LLM applications. When combined with the AI Gateway, you get the best of both worlds: LangSmith's detailed observability and the AI Gateway's advanced AI gateway features.

This integration allows you to:

* Track all LLM requests in LangSmith while routing through the AI Gateway
* Use the AI Gateway's 3,000+ LLMs with LangSmith observability
* Implement advanced features like caching, fallbacks, and load balancing
* Maintain detailed traces and analytics in both platforms

## Quick Start Integration

Since the AI Gateway provides an OpenAI-compatible API, integrating with LangSmith is straightforward using LangSmith's OpenAI wrapper.

### Installation

```bash theme={"system"}
pip install langsmith
```

### Basic Setup

<Note>
  This integration automatically logs requests to both LangSmith and the AI Gateway, giving you observability data in both platforms.
</Note>

## Using AI Gateway Features with LangSmith

### 1. LLM Integrations

LLM Integrations in the AI Gateway allow you to securely manage API keys and set usage limits. Use them with LangSmith for better security:

### 2. Multiple Providers

Switch between 3,000+ LLMs while maintaining LangSmith observability:

<Tabs>
  <Tab title="OpenAI">
    ```python theme={"system"}
    client = wrap_openai(Portkey(api_key="YOUR_PORTKEY_API_KEY"))

    response = client.chat.completions.create(
        model="@openai-integration/gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    ```
  </Tab>

  <Tab title="Anthropic">
    ```python theme={"system"}
    client = wrap_openai(Portkey(api_key="YOUR_PORTKEY_API_KEY"))

    response = client.chat.completions.create(
        model="@anthropic-integration/claude-3-sonnet-20240229",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    ```
  </Tab>

  <Tab title="Azure OpenAI">
    ```python theme={"system"}
    client = wrap_openai(Portkey(api_key="YOUR_PORTKEY_API_KEY"))

    response = client.chat.completions.create(
        model="@azure-openai-integration/gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    ```
  </Tab>
</Tabs>

### 3. Advanced Routing with Configs

Use the AI Gateway's config system for advanced features while tracking in LangSmith:

Example config for fallback between providers:

```json theme={"system"}
{
  "strategy": {
    "mode": "fallback"
  },
  "targets": [
    {
      "integration": "@openai-integration",
      "override_params": {"model": "gpt-4o"}
    },
    {
      "integration": "@anthropic-integration",
      "override_params": {"model": "claude-3-opus-20240229"}
    }
  ]
}
```

### 4. Caching for Cost Optimization

Enable caching to reduce costs while maintaining full observability:

### 5. Custom Metadata and Tracing

Add custom metadata visible in both LangSmith and the AI Gateway:

<CardGroup cols={3}>
  <Card title="Fallbacks" icon="life-ring" href="/docs/aigw/product/ai-gateway/fallbacks">
    Automatically switch to backup targets if the primary target fails.
  </Card>

  <Card title="Conditional Routing" icon="route" href="/docs/aigw/product/ai-gateway/conditional-routing">
    Route requests to different targets based on specified conditions.
  </Card>

  <Card title="Load Balancing" icon="key" href="/docs/aigw/product/ai-gateway/load-balancing">
    Distribute requests across multiple targets based on defined weights.
  </Card>

  <Card title="Caching" icon="database" href="/docs/aigw/product/ai-gateway/cache-simple-and-semantic">
    Enable caching of responses to improve performance and reduce costs.
  </Card>

  <Card title="Smart Retries" icon="database" href="/docs/aigw/product/ai-gateway/automatic-retries">
    Automatic retry handling with exponential backoff for failed requests
  </Card>

  <Card title="Budget Limits" icon="shield-check" href="/docs/aigw/product/policies/budget-limits">
    Set and manage budget limits across teams and departments. Control costs with granular budget limits and usage tracking.
  </Card>
</CardGroup>

## Observability Features

With this integration, you get:

### In LangSmith:

* Request/response logging
* Latency tracking
* Token usage analytics
* Cost calculation
* Trace visualization

### In the AI Gateway:

* Request logs with provider details
* Advanced analytics across providers
* Cost tracking and budgets
* Performance metrics
* Custom dashboards
* Token usage analytics

## Migration Guide

If you're already using LangSmith with OpenAI, migrating to use the AI Gateway is simple:

<CodeGroup>
  ```python Before theme={"system"}
  from openai import OpenAI
  from langsmith.wrappers import wrap_openai

  client = wrap_openai(OpenAI(api_key="YOUR_OPENAI_KEY"))
  ```
</CodeGroup>

## Next Steps

* [Create LLM Integrations](/docs/product/integrations) for secure API key management
* [Build Configs](/docs/aigw/product/ai-gateway/configs) for advanced routing
* [Set up Guardrails](/docs/aigw/product/guardrails) for content filtering
* [Implement Caching](/docs/aigw/product/ai-gateway/cache-simple-and-semantic) for cost optimization

## Resources

* [LangSmith Documentation](https://docs.smith.langchain.com/)
* [AI Gateway Guide](/docs/aigw/product/ai-gateway)

<Note>
  For enterprise support and custom features, contact our \[enterprise team]\([https://www.paloaltonetworks.com/ai-security/ai-gateway#:\~:text=See%20Prisma%20AIRS%20AI%20Gateway%20in%20Action](https://www.paloaltonetworks.com/ai-security/ai-gateway#:~:text=See%20Prisma%20AIRS%20AI%20Gateway%20in%20Action)
</Note>


## Related topics

- [Complete Logs Export](/docs/aigw/product/enterprise-offering/otel/complete-logs.md)
- [OpenTelemetry(OTel) Export](/docs/aigw/product/enterprise-offering/otel/otel.md)
- [LangGraph](/docs/aigw/integrations/agents/langgraph.md)
- [OpenTelemetry for LLM Observability](/docs/aigw/product/observability/opentelemetry.md)