This feature is available on all Portkey plans.

You can capture this feedback on a request or conversation level and analyze it by adding meta data to the relevant request.

Adding Feedback to Requests

1. Find the `trace-id`

Portkey adds trace ids to all incoming requests. You can find this in the x-portkey-trace-id response header.

To use your own trace IDs, send them as part of the request headers - Adding a trace ID to your requests

2. Add feedback

You can append feedback to a request through the SDKs or the REST API.

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
});

// Add the feedback
portkey.feedback.create({
    traceID: "your trace id",
    value: 5, // Integer between -10 and 10
    weight: 1, // Optional
    metadata: {
        ... // Pass any additional context here like comments, _user and more
    }
})

The Payload takes the following keys: traceID/trace_id, value, weight, metadata

KeyRequired?DescriptionType
trace_id / traceID

Required
The trace id on which the feedback will be loggedstring
value

Required
Feedback valueinteger between [-10,10]
weight

Optional
Add weight value to feedback value. Helpful if you’re collecting multiple feedback for a single tracefloat between [0,1], Default = 1.0
metadata

Optional
JSON string of any metadata you want to send along with the feedback._user, _organisation, _prompt and _environment are special fields indexed by defaultstring

Examples

A simple & effective feedback from the user is a thumbs up or thumbs down. Just set value to 1 for 👍 and -1 for 👎. Weight would be default 1.0.

portkey.feedback.create({
    traceID: "your trace id",
    value: 1
})

Other Ideas for collecting feedback

  • Business metrics make for great feedback. If you’re generating an email, the email being sent out could be a positive feedback metric. The level of editing could indicate the value.
  • When a user retries a generation, store the negative feedback since something probably went wrong. Use a lower weight for this feedback since it could be circumstantial.

Feedback Analytics

You can see the Feedback Count and Value: Weight pairs for each trace-id on the logs page. You can also view the feedback details on Analytics and on the Prompt Eval tabs.