Tracking LLM Costs Per User with Portkey

LLM runs are expensive. As your application scales, tracking costs per user, team, or workflow becomes essential to maintaining efficiency and budgeting effectively.

Traditional solutions only provide API key-level tracking, requiring complex custom systems for detailed analysis. Portkey solves this with metadata-based tracking, allowing you to monitor and allocate costs seamlessly.

​

Why Track LLM Costs Per User?

Consider these scenarios:

1. Your SaaS platform serves thousands of users—how do you track and bill their usage fairly?
2. Your enterprise team needs cost transparency across different departments—how do you allocate expenses?
3. Your application has multiple features leveraging LLMs—how much is each feature costing you?

With Portkey, you can track costs per user using two methods:

1. Portkey Dashboard: Use metadata filters to analyze costs visually.
2. Analytics API: Integrate with your systems to display real-time cost insights in your app.

This guide walks you through both approaches.

​

Step 1: Attach User Metadata to LLM Requests

Portkey allows you to attach metadata (key-value pairs) to each request, enabling cost breakdowns per user.

Portkey’s metadata accepts a JSON object of key:value pair. Each metadata value should be a string (max length: 128 characters). Here’s how to implement it:

Each metadata field in your request helps categorize and track costs. You can inject these values dynamically from your application context.

​

Step 2: View User Costs in the Portkey Dashboard

Portkey’s dashboard provides a live view of costs broken down by metadata filters.

1. Open Portkey Dashboard
2. In the Search Bar, select >Meta
3. Add a metadata filter, e.g., _user = customer_12345

You can also combine filters (e.g., _user + env) and adjust the time range for historical cost tracking.

This gives you an instant breakdown of LLM expenses per user or team.

​

Step 3: Fetch Cost Data Programmatically via the Analytics API

For deeper integrations, the Analytics API enables real-time cost tracking inside your own application. This is useful for:

• User-facing billing dashboards (show users their LLM usage in real time)
• Automated cost monitoring (trigger alerts when a user’s spending exceeds a threshold)
• Enterprise reporting (export data for budget forecasting)

Understanding the Analytics API

The API provides comprehensive cost analytics data across any metadata dimension you’ve configured. You can query historical data, aggregate costs across different timeframes, and access detailed metrics for each metadata value. Here’s what you can access through the API:

"data": [
  {
    "metadata_value": "kreacher",
    "requests": 4344632,
    "cost": 3887.3066999996863,
    "avg_tokens": 447.3689256075083,
    "avg_weighted_feedback": 4.2,
    "requests_with_feedback": 10,
    "last_seen": "2025-02-03T07:19:30.000Z",
    "object": "analytics-group"
  },
  {
    ...more such objects
  }

These metrics provide insights into costs, usage patterns, and efficiency. The response includes:

• Total requests and costs per metadata value
• Average token usage for optimization analysis
• User feedback metrics for quality assessment
• Timestamp data for temporal analysis

​

Step 4: Tracking User Costs using Portkey’s Analytics API

Before making your first API call, you’ll need to obtain an API key from the Portkey Dashboard. This key requires analytics scope access, which you can configure in your API key settings.

The API follows RESTful principles and accepts standard HTTP requests. Here’s how to get started:

1. Replace the api key in the x-portkey-api-key header.
2. In the url replace {metadataKey} with the metadata key you want to track costs for.

That’s all you need to do the get the resonse data with the cost metric for the metadata key you are tracking.

How Developers & Teams Use This API

• CTOs: Gain visibility into LLM costs across teams.
• DevOps: Monitor and optimize token usage.
• Product Teams: Track costs by feature to identify inefficiencies.
• Finance Teams: Automate cost allocation and reporting.

​

Step 5: Automate User Cost Tracking

Once you’ve integrated metadata tagging and the Analytics API, you can:

• Trigger alerts when users exceed spending limits
• Embed LLM cost breakdowns into your SaaS billing dashboard
• Optimize feature costs by analyzing usage patterns

​

Next Steps

1. Set Up Budget Limits for API Keys
2. Implement Fallback Strategies for Cost Control
3. Explore Portkey’s Prompt Management

Need enterprise-grade cost tracking? Contact Sales for custom analytics solutions.