Milvus - Portkey Docs

Milvus is an open-source vector database built for GenAI applications. It is built to be performant and scale to tens of billions of vectors with minimal performance loss. The Prisma AIRS AI Gateway provides a proxy to Milvus - you can log your Milvus requests and manage auth for Qdrant clusters on the AI Gateway.

Using Milvus with the AI Gateway

The AI Gateway provides a consistent API to interact with models from various providers. To integrate Milvus with the AI Gateway:

1. Install the OpenAI SDK

Add the OpenAI SDK to your application to interact with Milvus through the AI Gateway.

NodeJS
Python

npm install openai --save

pip install openai

2. Create a New Integration

To use Milvus with the AI Gateway, get your Milvus API key and create a new Milvus integration on the AI Gateway.

OpenAI Python SDK
OpenAI Node SDK

from openai import OpenAI

client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={
        "x-portkey-provider": "milvus",
        "x-portkey-custom-host": "MILVUS_HOST",  # Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com
    }
)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "PORTKEY_API_KEY",
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: {
      "x-portkey-provider": "milvus",
      "x-portkey-custom-host": "MILVUS_HOST",  // Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com
  },
});

3. Use the OpenAI SDK to interact with Milvus

cURL
OpenAI NodeJS
OpenAI Python

curl --location --request POST 'https://aigw.portkey.ai/v1/v2/vectordb/collections/list' \
--header 'x-portkey-custom-host: https://in03-34d7b37f7de12c7.serverless.gcp-us-west1.cloud.zilliz.com' \
--header 'x-portkey-provider: MILVUS_PROVIDER' \
--header 'Authorization: Bearer PORTKEY_API_KEY'

import OpenAI from 'openai'; // We're using the v4 SDK

const openai = new OpenAI({
  apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
  baseURL: "https://aigw.portkey.ai/v1",
  defaultHeaders: {
      "x-portkey-provider": "@MILVUS_PROVIDER",
      // defaults to process.env["PORTKEY_API_KEY"]
      "x-portkey-custom-host": "MILVUS_HOST" // Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com,
  }
});

response = openai.post(
    url="v2/vectordb/collections/list", # Replace with the endpoint you want to call
)

print(response)

from openai import OpenAI

openai = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://aigw.portkey.ai/v1",
    default_headers={
        "x-portkey-provider": "milvus",
        "x-portkey-custom-host": "MILVUS_HOST",  # Replace with your Milvus host example: https://in03-34d7b37f7ee12c7.serverless.gcp-us-west1.cloud.zilliz.com
    }
)

response = openai.post(
    url="v2/vectordb/collections/list", # Replace with the endpoint you want to call
)

print(response)

Last modified on September 15, 2026