> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama

> Integrate Ollama-hosted models with Prisma AIRS AI Gateway for local LLM deployment with full observability.

The AI Gateway provides a robust gateway to facilitate the integration of your **locally hosted models through Ollama**.

<AccordionGroup>
  <Accordion title="Local Setup (npm or Docker)">
    First, install the Gateway locally:

    <CodeGroup>
      ```bash npx theme={"system"}
      npx @portkey-ai/gateway
      ```

      ```bash Docker theme={"system"}
      docker run -d -p 8787:8787 portkeyai/gateway:latest
      ```
    </CodeGroup>

    Then, connect to your local Ollama instance:
  </Accordion>
</AccordionGroup>

## Integration Steps

<Steps>
  <Step title="Expose your Ollama API">
    Expose your Ollama API using a tunneling service like [ngrok](https://ngrok.com/) or make it publicly accessible. Skip this if you're self-hosting the Gateway.

    For using Ollama with ngrok, here's a [useful guide](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-use-ollama-with-ngrok):

    ```sh theme={"system"}
    ngrok http 11434 --host-header="localhost:11434"
    ```
  </Step>

  <Step title="Add to Model Catalog">
    1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
    2. Enable **"Local/Privately hosted provider"** toggle
    3. Select **Ollama** as the provider type
    4. Enter your Ollama URL in **Custom Host**: `https://your-ollama.ngrok-free.app`
    5. Name your provider (e.g., `my-ollama`)

    <Card title="Complete Setup Guide" icon="book" href="/docs/aigw/product/model-catalog">
      See all setup options
    </Card>
  </Step>

  <Step title="Use in Your Application">
    **Or use custom host directly:**
  </Step>
</Steps>

<Note>
  **Important:** For Ollama integration, you only need to pass the base URL to `customHost` **without** the version identifier (such as `/v1`) - the AI Gateway handles the rest!
</Note>

***

## Supported Models

Ollama supports a wide range of models including:

* Llama 3, Llama 3.1, Llama 3.2
* Mistral, Mixtral
* Gemma, Gemma 2
* Phi-3
* Qwen 2
* And many more!

Check [Ollama's model library](https://ollama.com/library) for the complete list.

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Gateway Configs" icon="sliders" href="/docs/aigw/product/ai-gateway">
    Add retries, timeouts, and fallbacks
  </Card>

  <Card title="Observability" icon="chart-line" href="/docs/aigw/product/observability">
    Monitor your Ollama requests
  </Card>

  <Card title="Custom Host Guide" icon="server" href="/docs/aigw/product/ai-gateway/universal-api#integrating-local-or-private-models">
    Learn more about custom host setup
  </Card>

  <Card title="BYOLLM Guide" icon="book" href="/docs/aigw/integrations/llms/byollm">
    Complete guide for private LLMs
  </Card>
</CardGroup>


## Related topics

- [Universal API](/docs/aigw/product/ai-gateway/universal-api.md)
- [Custom hosts](/docs/aigw/product/ai-gateway/custom-hosts.md)
- [Anthropic Computer Use](/docs/aigw/integrations/libraries/anthropic-computer-use.md)
- [Enterprise Gateway](/docs/aigw/changelog/enterprise.md)
- [Overview](/docs/aigw/integrations/llms.md)