> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Triton Inference Server

> Integrate Triton-hosted custom models with Prisma AIRS AI Gateway for production observability and reliability.

The AI Gateway provides a robust platform to observe, govern, and manage your **locally** or **privately** hosted custom models using Triton Inference Server.

<Info>
  Here's the official [Triton Inference Server documentation](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/getting_started/quickstart.html) for more details.
</Info>

## Integration Steps

<Steps>
  <Step title="Expose your Triton Server">
    Expose your Triton server using a tunneling service like [ngrok](https://ngrok.com/) or make it publicly accessible. Skip this if you're self-hosting the Gateway.

    ```sh theme={"system"}
    ngrok http 8000 --host-header="localhost:8080"
    ```
  </Step>

  <Step title="Add to Model Catalog">
    1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
    2. Enable **"Local/Privately hosted provider"** toggle
    3. Select **Triton** as the provider type
    4. Enter your Triton server URL in **Custom Host**: `http://localhost:8000/v2/models/mymodel`
    5. Add authentication headers if needed
    6. Name your provider (e.g., `my-triton`)

    <Card title="Complete Setup Guide" icon="book" href="/docs/aigw/product/model-catalog">
      See all setup options
    </Card>
  </Step>

  <Step title="Use in Your Application">
    **Or use custom host directly:**
  </Step>
</Steps>

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Gateway Configs" icon="sliders" href="/docs/aigw/product/ai-gateway">
    Add retries, timeouts, and fallbacks
  </Card>

  <Card title="Observability" icon="chart-line" href="/docs/aigw/product/observability">
    Monitor your Triton deployments
  </Card>

  <Card title="Custom Host Guide" icon="server" href="/docs/aigw/product/ai-gateway/universal-api#integrating-local-or-private-models">
    Learn more about custom host setup
  </Card>

  <Card title="BYOLLM Guide" icon="book" href="/docs/aigw/integrations/llms/byollm">
    Complete guide for private LLMs
  </Card>
</CardGroup>


## Related topics

- [Supported Providers](/docs/aigw/api-reference/inference-api/supported-providers.md)
- [Inference.net](/docs/aigw/integrations/llms/inference.net.md)
- [Remote MCP](/docs/aigw/product/ai-gateway/remote-mcp.md)
- [Overview](/docs/aigw/integrations/llms.md)
- [Unified Batch Inference](/docs/aigw/product/ai-gateway/batches.md)