Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including all the text generation models supported by Huggingface’s Inference endpoints.

With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a virtual key system.

Provider Slug. huggingface

Portkey SDK Integration with Huggingface

Portkey provides a consistent API to interact with models from various providers. To integrate Huggingface with Portkey:

1. Install the Portkey SDK

Add the Portkey SDK to your application to interact with Huggingface’s API through Portkey’s gateway.

npm install --save portkey-ai

2. Initialize Portkey with the Virtual Key

To use Huggingface with Portkey, get your Huggingface Access token from here, then add it to Portkey to create the virtual key.

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    virtualKey: "VIRTUAL_KEY", // Your Huggingface Virtual Key
    huggingfaceBaseUrl: "HUGGINGFACE_DEDICATED_URL" // Optional: Use this if you have a dedicated server hosted on Huggingface
})

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    virtualKey: "VIRTUAL_KEY", // Your Huggingface Virtual Key
    huggingfaceBaseUrl: "HUGGINGFACE_DEDICATED_URL" // Optional: Use this if you have a dedicated server hosted on Huggingface
})

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    virtual_key="VIRTUAL_KEY",   # Replace with your virtual key for Huggingface
    huggingface_base_url="HUGGINGFACE_DEDICATED_URL" # Optional: Use this if you have a dedicated server hosted on Huggingface
)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

client = OpenAI(
    api_key="HUGGINGFACE_ACCESS_TOKEN",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        api_key="PORTKEY_API_KEY",
        provider="huggingface",
        huggingface_base_url="HUGGINGFACE_DEDICATED_URL"
    )
)

import OpenAI from "openai";
import { PORTKEY_GATEWAY_URL, createHeaders } from "portkey-ai";

const client = new OpenAI({
  apiKey: "HUGGINGFACE_ACCESS_TOKEN",
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "huggingface",
    apiKey: "PORTKEY_API_KEY",
    huggingfaceBaseUrl: "HUGGINGFACE_DEDICATED_URL"
  }),
});

3. Invoke Chat Completions with Huggingface

Use the Portkey instance to send requests to Huggingface. You can also override the virtual key directly in the API call if needed.

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'meta-llama/Meta-Llama-3.1-8B-Instruct', // make sure your model is hot
});

console.log(chatCompletion.choices[0].message.content);

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'meta-llama/Meta-Llama-3.1-8B-Instruct', // make sure your model is hot
});

console.log(chatCompletion.choices[0].message.content);

chat_completion = portkey.chat.completions.create(
    messages= [{ "role": 'user', "content": 'Say this is a test' }],
    model= 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot
)

print(chat_completion.choices[0].message.content)

chat_completion = client.chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot
)

print(chat_completion.choices[0].message.content)

async function main() {
    const chatCompletion = await client.chat.completions.create({
        model: "meta-llama/meta-llama-3.1-8b-instruct", // make sure your model is hot
        messages: [{ role: "user", content: "How many points to Gryffindor?" }],
    });
    console.log(chatCompletion.choices[0].message.content);
}

main();

chat_completion = client.chat.completions.create(
    messages = [{ "role": 'user', "content": 'Say this is a test' }],
    model = 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot
)

print(chat_completion.choices[0].message.content)

Using Virtual Keys

Virtual Keys serve as Portkey’s unified authentication system for all LLM interactions, simplifying the use of multiple providers and Portkey features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model:

Navigate to Virtual Keys in your Portkey dashboard
Click “Add Key” and enable the “Local/Privately hosted provider” toggle
Configure your deployment:
- Select the matching provider API specification (typically OpenAI)
- Enter your model’s base URL in the Custom Host field
- Add required authentication headers and their values
Click “Create” to generate your virtual key

You can now use this virtual key in your requests:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY"

async function main() {
  const response = await client.chat.completions.create({
    messages: [{ role: "user", content: "Bob the builder.." }],
    model: "your-self-hosted-model-name",
  });

console.log(response.choices[0].message.content);
})

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY"

async function main() {
  const response = await client.chat.completions.create({
    messages: [{ role: "user", content: "Bob the builder.." }],
    model: "your-self-hosted-model-name",
  });

console.log(response.choices[0].message.content);
})

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY"
)

response = portkey.chat.completions.create(
  model="your-self-hosted-model-name",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]

print(response)
)

For more information about managing self-hosted LLMs with Portkey, see Bring Your Own LLM.

Next Steps

The complete list of features supported in the SDK are available on the link below.

SDK

You’ll find more information in the relevant sections:

Groq Inference.net

On this page

Portkey SDK Integration with Huggingface
1. Install the Portkey SDK
2. Initialize Portkey with the Virtual Key
3. Invoke Chat Completions with Huggingface
Using Virtual Keys
Next Steps

Ecosystem

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

Hugging Face

Portkey SDK Integration with Huggingface

1. Install the Portkey SDK

2. Initialize Portkey with the Virtual Key

3. Invoke Chat Completions with Huggingface

Using Virtual Keys

Next Steps

SDK

Ecosystem

LLM Integrations

Cloud Platforms

Guardrails

Plugins

Vector Databases

Agents

AI Apps

Libraries

Tracing Providers

​Portkey SDK Integration with Huggingface

​1. Install the Portkey SDK

​2. Initialize Portkey with the Virtual Key

​3. Invoke Chat Completions with Huggingface

​Using Virtual Keys

​Next Steps

SDK

Portkey SDK Integration with Huggingface

1. Install the Portkey SDK

2. Initialize Portkey with the Virtual Key

3. Invoke Chat Completions with Huggingface

Using Virtual Keys

Next Steps