Github

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on Github Models Marketplace.

​

Portkey SDK Integration with Github Models

Portkey provides a consistent API to interact with models from various providers. To integrate Github Models with Portkey:

​

1. Install the Portkey SDK

• NodeJS
• Python

npm install --save portkey-ai

​

2. Initialize Portkey with Github Models

To use Github with Portkey, get your API key from here, then add it to Portkey to create the virtual key.

• NodeJS SDK
• Python SDK

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    virtualKey: "GITHUB_VIRTUAL_KEY" // Your Github Models virtual key
})

​

3. Invoke Chat Completions

• NodeJS SDK
• Python SDK

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'Phi-3-small-128k-instruct',
});

console.log(chatCompletion.choices);

​

Supported Models

Portkey supports all the models (both Chat/completion and Embeddings capabilities) on the Github Models marketplace.

​

Next Steps

The complete list of features supported in the SDK are available on the link below.

You’ll find more information in the relevant sections:

1. Add metadata to your requests
2. Add gateway configs to your requests requests
3. Tracing Github requests
4. Setup a fallback from OpenAI to Github