This feature is available on all Portkey plans.
You can capture this feedback on a request or conversation level and analyze it by adding meta data to the relevant request.

​
Adding Feedback to Requests

​
1. Find the `trace-id`

Portkey adds trace ids to all incoming requests. You can find this in the x-portkey-trace-id response header. To use your own trace IDs, send them as part of the request headers - Adding a trace ID to your requests

​
2. Add feedback

You can append feedback to a request through the SDKs or the REST API. 
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
});

// Add the feedback
portkey.feedback.create({
    traceID: "your trace id",
    value: 5, // Integer between -10 and 10
    weight: 1, // Optional
    metadata: {
        ... // Pass any additional context here like comments, _user and more
    }
})
The Payload takes the following keys: traceID/trace_id, value, weight, metadata
KeyRequired?DescriptionType
trace_id / traceID RequiredThe trace id on which the feedback will be loggedstring
value RequiredFeedback valueinteger between [-10,10]
weight OptionalAdd weight value to feedback value. Helpful if you’re collecting multiple feedback for a single tracefloat between [0,1], Default = 1.0
metadata OptionalJSON string of any metadata you want to send along with the feedback._user, _organisation, _prompt and _environment are special fields indexed by defaultstring

​
Examples

A simple & effective feedback from the user is a thumbs up or thumbs down. Just set value to 1 for 👍 and -1 for 👎. Weight would be default 1.0. 
portkey.feedback.create({
    traceID: "your trace id",
    value: 1
})

​
Other Ideas for collecting feedback

  • Business metrics make for great feedback. If you’re generating an email, the email being sent out could be a positive feedback metric. The level of editing could indicate the value.
  • When a user retries a generation, store the negative feedback since something probably went wrong. Use a lower weight for this feedback since it could be circumstantial.

​
Feedback Analytics

You can see the Feedback Count and Value: Weight pairs for each trace-id on the logs page. You can also view the feedback details on Analytics and on the Prompt Eval tabs.