> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Llama Agents by Llamaindex

> Use Prisma AIRS AI Gateway with Llama Agents to take your AI Agents to production

## Getting Started

### 1. Install the required packages:

```sh theme={"system"}
pip install -qU llama-agents llama-index
```

### 2. Configure your Llama Index LLM objects:

```py theme={"system"}
from llama_index.llms.openai import OpenAI

gpt_4o = OpenAI(
    api_key="YOUR_PORTKEY_API_KEY",
    api_base="https://aigw.portkey.ai/v1",
    model="@YOUR_OPENAI_PROVIDER/gpt-4o",
)
```

That's all you need to do to use the AI Gateway with Llama Index agents. Execute your agents and visit the [Strata Cloud Manager](https://stratacloudmanager.paloaltonetworks.com/) to observe your Agent's activity.

## Make your agents Production-ready with AI Gateway

The AI Gateway makes your Llama Index agents reliable, robust, and production-grade with its observability suite. Seamlessly integrate 3,000+ LLMs with your Llama Index agents, implement fallbacks, gain granular insights into agent performance and costs, and continuously optimize your AI operations—all with just 2 lines of code.

Let's dive deep! Let's go through each of the use cases!

### 1. [Interoperability](/docs/aigw/product/ai-gateway/universal-api)

Easily switch between 3,000+ LLMs. Call various LLMs such as Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock, and many more by changing the provider slug prefixed to the `model` name.

<Tabs>
  <Tab title="OpenAI to Azure OpenAI">
    If you are using OpenAI, your code would look like this:

    ```py theme={"system"}
    llm = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        api_base="https://aigw.portkey.ai/v1",
        model="@YOUR_OPENAI_PROVIDER/gpt-4o",
    )
    ```

    To switch to Azure as your provider, add your Azure details to the Model Catalog and create an integration ([here's how](/docs/aigw/integrations/llms/azure-openai/azure-openai)), then point the model at that provider slug.

    ```py theme={"system"}
    llm = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        api_base="https://aigw.portkey.ai/v1",
        model="@YOUR_AZURE_PROVIDER/gpt-4o",
    )
    ```
  </Tab>

  <Tab title="Anthropic to AWS Bedrock">
    If you are using Anthropic, your code would look like this:

    ```py theme={"system"}
    llm = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        api_base="https://aigw.portkey.ai/v1",
        model="@YOUR_ANTHROPIC_PROVIDER/claude-3-5-sonnet-20240620",
    )
    ```

    To switch to AWS Bedrock as your provider, add your AWS Bedrock details to the Model Catalog and create an integration ([here's how](/docs/aigw/integrations/llms/bedrock/aws-bedrock)), then point the model at that provider slug.

    ```py theme={"system"}
    llm = OpenAI(
        api_key="YOUR_PORTKEY_API_KEY",
        api_base="https://aigw.portkey.ai/v1",
        model="@YOUR_BEDROCK_PROVIDER/anthropic.claude-3-5-sonnet-20240620-v1:0",
    )
    ```
  </Tab>
</Tabs>

### 2. [Reliability](/docs/aigw/product/ai-gateway)

Agents are *brittle*. Long agentic pipelines with multiple steps can fail at any stage, disrupting the entire process. The AI Gateway solves this by offering built-in **fallbacks** between different LLMs or providers, **load-balancing** across multiple instances or API keys, and implementing automatic **retries** and request **timeouts**. This makes your agents more reliable and resilient.

Here's how you can implement these features using the AI Gateway's config

```json theme={"system"}
{
  "retry": {
    "attempts": 5
  },
  "strategy": {
    "mode": "loadbalance"
  },
  "targets": [
    { "provider": "@openai-prod" },
    { "provider": "@anthropic-prod" }
  ]
}
```

### 3. [Metrics](/docs/aigw/product/observability)

Agent runs can be costly. Tracking agent metrics is crucial for understanding the performance and reliability of your AI agents. Metrics help identify issues, optimize runs, and ensure that your agents meet their intended goals.

The AI Gateway automatically logs comprehensive metrics for your AI agents, including **cost**, **tokens used**, **latency**, etc. Whether you need a broad overview or granular insights into your agent runs, the AI Gateway's customizable filters provide the metrics you need. For agent-specific observability, add `Trace-id` to the request headers for each agent.

```py theme={"system"}
llm = OpenAI(
    api_key="YOUR_PORTKEY_API_KEY",
    api_base="https://aigw.portkey.ai/v1",
    model="@YOUR_ANTHROPIC_PROVIDER/claude-3-5-sonnet-20240620",
    default_headers={
        "x-portkey-trace-id": "research_agent1",  # Add individual trace-id for your agent analytics
    },
)
```

### 4. [Logs](/docs/aigw/product/observability/logs)

Agent runs are complex. Logs are essential for diagnosing issues, understanding agent behavior, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes.

The AI Gateway offers comprehensive logging features that capture detailed information about every action and decision made by your AI agents. Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata.

### 5. [Traces](/docs/aigw/product/observability/traces)

With traces, you can see each agent run granularly on the AI Gateway. Tracing your LlamaIndex agent runs helps in debugging, performance optimzation, and visualizing how exactly your agents are running.

### Using Traces in LlamaIndex Agents

Send a `x-portkey-trace-id` header with every call your agent makes. All requests that share a trace ID are grouped into a single trace in the AI Gateway.

```py theme={"system"}
from llama_index.llms.openai import OpenAI

llm = OpenAI(
    api_key="YOUR_PORTKEY_API_KEY",
    api_base="https://aigw.portkey.ai/v1",
    model="@YOUR_OPENAI_PROVIDER/gpt-4o",
    default_headers={"x-portkey-trace-id": "research_agent1"},
)
```

With the AI Gateway tracing, you can encapsulate the complete execution of your agent workflow.

### 6. [Continuous Improvement](/docs/aigw/product/observability/feedback)

Improve your Agent runs by capturing qualitative & quantitative user feedback on your requests. The AI Gateway's Feedback APIs provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app. You can capture this feedback on a request or conversation level and analyze it by adding meta data to the relevant request.

### 7. [Caching](/docs/aigw/product/ai-gateway/cache-simple-and-semantic)

Agent runs are time-consuming and expensive due to their complex pipelines. Caching can significantly reduce these costs by storing frequently used data and responses. The AI Gateway offers a built-in caching system that stores past responses, reducing the need for agent calls saving both time and money.

Choose between `simple` and `semantic` caching:

```json theme={"system"}
{
  "cache": {
    "mode": "semantic"
  }
}
```

### 8. [Security & Compliance](/docs/aigw/product/enterprise-offering/security)

Set budget limits on provider API keys and implement fine-grained user roles and permissions for both the app and the AI Gateway APIs.

***

## [AI Gateway Config](/docs/aigw/product/ai-gateway/configs)

Many of these features are driven by the AI Gateway's Config architecture. Strata Cloud Manager simplifies creating, managing, and versioning your Configs.

For more information on using these features and setting up your Config, please refer to the AI Gateway documentation.


## Related topics

- [LlamaIndex (Python)](/docs/aigw/integrations/libraries/llama-index-python.md)
- [Tracing](/docs/aigw/product/observability/traces.md)
- [Overview](/docs/aigw/integrations/libraries.md)
- [Agentic Usage](/docs/aigw/api-reference/inference-api/agentic-usage.md)
- [MLflow Tracing](/docs/aigw/integrations/tracing-providers/ml-flow.md)