Skip to main content
The AI Gateway provides a robust platform to observe, govern, and manage your locally or privately hosted custom models using Triton Inference Server.
Here’s the official Triton Inference Server documentation for more details.

​
Integration Steps

1

Expose your Triton Server

Expose your Triton server using a tunneling service like ngrok or make it publicly accessible. Skip this if you’re self-hosting the Gateway.
2

Add to Model Catalog

  1. Go to Model Catalog → Add Provider
  2. Enable “Local/Privately hosted provider” toggle
  3. Select Triton as the provider type
  4. Enter your Triton server URL in Custom Host: http://localhost:8000/v2/models/mymodel
  5. Add authentication headers if needed
  6. Name your provider (e.g., my-triton)

Complete Setup Guide

See all setup options
3

Use in Your Application

Or use custom host directly:

​
Next Steps

Gateway Configs

Add retries, timeouts, and fallbacks

Observability

Monitor your Triton deployments

Custom Host Guide

Learn more about custom host setup

BYOLLM Guide

Complete guide for private LLMs
Last modified on September 15, 2026