Skip to main content
The AI Gateway provides a robust gateway to facilitate the integration of your locally hosted models through LocalAI.

​
Integration Steps

1

Expose your LocalAI Server

Ensure your LocalAI API is externally accessible. If running on http://localhost, use a tool like ngrok to create a public URL.
2

Add to Model Catalog

  1. Go to Model Catalog → Add Provider
  2. Enable “Local/Privately hosted provider” toggle
  3. Select OpenAI as the provider type (LocalAI follows OpenAI API schema)
  4. Enter your LocalAI URL with /v1 in Custom Host: https://your-localai.ngrok-free.app/v1
  5. Name your provider (e.g., my-localai)

Complete Setup Guide

See all setup options
3

Use in Your Application

Or use custom host directly:
Important:
  • Don’t forget to include the version identifier (/v1) in the Custom Host URL
  • The AI Gateway supports all endpoints that adhere to the OpenAI specification

​
LocalAI Endpoints Supported

​
Next Steps

Gateway Configs

Add retries, timeouts, and fallbacks

Observability

Monitor your LocalAI requests

Custom Host Guide

Learn more about custom host setup

BYOLLM Guide

Complete guide for private LLMs
Last modified on September 15, 2026