Building Enterprise LLM Apps with .NET

.NET is Microsoft’s battle-tested framework trusted by Fortune 500 companies. It’s now easier than ever to build LLM apps. You get:

Battle-Tested SecurityBuilt-in identity management, secret rotation, and compliance standards
Production PerformanceHigh-throughput processing with advanced memory management
Azure IntegrationSeamless Azure OpenAI and Active Directory support

Combined with Portkey’s enterprise features, you get everything needed for mission-critical LLM deployments. Monitor costs, ensure reliability, maintain compliance, and scale with confidence.

Portkey Features

Complete ObservabilityMonitor costs, latency, and performance metrics
Provider FlexibilityRoute to 250+ LLMs (like Claude, Gemini, Llama, self-hosted etc.) without code changes
Smart CachingReduce costs & time by caching frequent requests
High ReliabilityAutomatic fallback and load balancing across providers
Prompt ManagementUse Portkey as a centralized hub to version, experiment with prompts, and call them using a single ID
Continuous ImprovementImprove your app by capturing and analyzing user feedback
Enterprise ReadyBudget controls, rate limits, model-provisioning, and role-based access

Supported Clients

ChatClient✅ Fully Supported
EmbeddingClient✅ Fully Supported
ImageClient🚧 Coming Soon
BatchClient🚧 Coming Soon
AudioClient🚧 Coming Soon

Implementation Overview

  1. Install OpenAI SDK
  2. Create Portkey client by extending OpenAI client
  3. Use the client in your application to make requests

1. Install the NuGet package

Add the OpenAI NuGet package to your .NET project:

dotnet add package OpenAI

2. Create Portkey Client Extension

The OpenAI package does not support directly modifying the base URL or passing additional headers. So, we write a simple function to extend OpenAI’s ChatClient or EmbeddingClient to create a new PortkeyClient.

using OpenAI;
using OpenAI.Chat;
using System.ClientModel;
using System.ClientModel.Primitives;

public static class PortkeyClient
{
    private class HeaderPolicy : PipelinePolicy
    {
        private readonly Dictionary<string, string> _headers;
        public HeaderPolicy(Dictionary<string, string> headers) => _headers = headers;

        public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int index)
        {
            foreach (var header in _headers) message.Request.Headers.Set(header.Key, header.Value);
            if (index < pipeline.Count) pipeline[index].Process(message, pipeline, index + 1);
        }

        public override ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int index)
        {
            Process(message, pipeline, index);
            return ValueTask.CompletedTask;
        }
    }

    public static ChatClient CreateChatClient(Dictionary<string, string> headers, string model)
    {
        var options = new OpenAIClientOptions { Endpoint = new Uri("https://api.portkey.ai/v1") };
        options.AddPolicy(new HeaderPolicy(headers), PipelinePosition.PerCall);
        return new OpenAIClient(new ApiKeyCredential("dummy"), options).GetChatClient(model);
    }
}

3. Use the Portkey Client

After creating the extension above, you can pass any Portkey supported headers directly while creating the new client.

// Define Portkey headers
var headers = new Dictionary<string, string> {
    // Required headers
    { "x-portkey-api-key", "..." },       // Your Portkey API key
    { "x-portkey-virtual-key", "..." },    // Virtual key for provider

    // Optional headers
    { "x-portkey-trace-id", "my-app" },       // Custom trace identifier
    { "x-portkey-config", "..." },            // Send Config ID
    // Add any other Portkey headers as needed
};

// Create client
var client = PortkeyClient.CreateChatClient(
    headers: headers,
    model: "gpt-4"
);

// Make request
var response = client.CompleteChat(new UserChatMessage("Yellow!"));
Console.WriteLine(response.Value.Content[0].Text);

While we show common headers here, you can pass any Portkey-supported headers to enable features like custom metadata, fallbacks, caching, retries, and more.

4. View Your Request in Portkey Logs

This request will now be logged on Portkey:

Chat Completions Example

Embedding Example

More Features

You can also use the PortkeyClient to send Async requests:

var completion = await client.CompleteChatAsync(new UserChatMessage("Hello!"));
Console.WriteLine(completion.Value.Content[0].Text);

Next Steps

Need Help?

Ping the Portkey team on our Developer Forum or email us at [email protected]