​

Overview

Portkey offers native integrations with Upstage for Node.js, Python, and REST APIs. By combining Portkey with Upstage, you can create production-grade AI applications with enhanced reliability, observability, and advanced features.

​

Getting Started

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    virtual_key="VIRTUAL_KEY"   # Replace with your virtual key for Upstage
)

from portkey_ai import Portkey, PORTKEY_GATEWAY_URL

portkey = Portkey(
    api_key="dummy",  # Replace with your Portkey API key
    base_url=PORTKEY_GATEWAY_URL,
    Authorization="UPSTAGE_API_KEY", # Replace with your Upstage API Key
    provider="upstage"
)

🔥 That’s it! You’ve integrated Portkey into your application with just a few lines of code. Now let’s explore making requests using the Portkey client.

​

Supported Models

​

Supported Endpoints and Parameters

​

Upstage Supported Features

​

Chat Completions

Generate chat completions using Upstage models through Portkey:

completion = portkey.chat.completions.create(
    messages=[{"role": "user", "content": "Say this is a test"}],
    model="solar-pro"
)

print(completion.choices[0].message.content)

​

Streaming

Stream responses for real-time output in your applications:

chat_complete = portkey.chat.completions.create(
    model="solar-pro",
    messages=[{"role": "user", "content": "Say this is a test"}],
    stream=True
)

for chunk in chat_complete:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

​

Function Calling

Leverage Upstage’s function calling capabilities through Portkey:

let tools = [{
    type: "function",
    function: {
        name: "getWeather",
        description: "Get the current weather",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state" },
                unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["location"]
        }
    }
}];

let response = await portkey.chat.completions.create({
    model: "solar-pro",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What's the weather like in Delhi - respond in JSON" }
    ],
    tools,
    tool_choice: "auto",
});

console.log(response.choices[0].finish_reason);

​

Embeddings

Generate embeddings for text using Upstage embedding models:

response = portkey.embeddings.create(
    input="Your text string goes here",
    model="embedding-query"
)

print(response.data[0].embedding)

​

Portkey’s Advanced Features

​

Track End-User IDs

Portkey allows you to track user IDs passed with the user parameter in Upstage requests, enabling you to monitor user-level costs, requests, and more:

response = portkey.chat.completions.create(
  model="solar-pro",
  messages=[{"role": "user", "content": "Say this is a test"}],
  user="user_123456"
)

When you include the user parameter in your requests, Portkey logs will display the associated user ID, as shown in the image below:

In addition to the user parameter, Portkey allows you to send arbitrary custom metadata with your requests. This powerful feature enables you to associate additional context or information with each request, which can be useful for analysis, debugging, or other custom use cases.

​

Using The Gateway Config

Here’s a simplified version of how to use Portkey’s Gateway Configuration:

config = {
  "strategy": {
    "mode": "conditional",
    "conditions": [
      {
        "query": { "metadata.user_plan": { "$eq": "paid" } },
        "then": "solar-pro"
      },
      {
        "query": { "metadata.user_plan": { "$eq": "free" } },
        "then": "gpt-3.5"
      }
    ],
    "default": "base-gpt4"
  },
  "targets": [
    {
      "name": "solar-pro",
      "virtual_key": "xx"
    },
    {
      "name": "gpt-3.5",
      "virtual_key": "yy"
    }
  ]
}

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    virtual_key="VIRTUAL_KEY",
    config=portkey_config
)

That’s it! Portkey seamlessly allows you to make your AI app more robust using built-in gateway features. Learn more about advanced gateway features:

​

Guardrails

Portkey’s AI gateway enables you to enforce input/output checks on requests by applying custom hooks before and after processing. Protect your user’s/company’s data by using PII guardrails and many more available on Portkey Guardrails:

{
	"virtual_key":"upstage-xxx",
	"before_request_hooks": [{
		"id": "input-guardrail-id-xx"
	}],
	"after_request_hooks": [{
		"id": "output-guardrail-id-xx"
	}]
}

​

Next Steps

The complete list of features supported in the SDK are available in our comprehensive documentation:

​

Limitations

For the most up-to-date information on supported features and endpoints, please refer to our API Reference.