Feedback

You can capture this feedback on a request or conversation level and analyze it by adding meta data to the relevant request.

Adding Feedback to Requests

1. Find the `trace-id`

The AI Gateway adds trace ids to all incoming requests. You can find this in the x-portkey-trace-id response header. To use your own trace IDs, send them as part of the request headers - Adding a trace ID to your requests

2. Add feedback

You can append feedback to a request through the SDKs or the REST API.

cURL

curl --location 'https://aigw.portkey.ai/v1/feedback' \
    --header 'Authorization: Bearer YOUR_API_KEY' \
    --header 'Content-Type: application/json' \
    --data '{
        "trace_id": "YOUR_TRACE_ID",
        "value": -10,
        "weight": 0.5,
        "metadata": {
            "text": "title was irrelevant",
            "_user": "fef653",
            "_organisation": "o9876",
            "_prompt": "test_prompt",
            "_environment": "production"
        }
    }'

The Payload takes the following keys: trace_id, value, weight, metadata

Key	Required?	Description	Type
trace_id	Required	The trace id on which the feedback will be logged	string
value	Required	Feedback value	integer between [-10,10]
weight	Optional	Add weight value to feedback value. Helpful if you’re collecting multiple feedback for a single trace	float between [0,1], Default = 1.0
metadata	Optional	JSON string of any metadata you want to send along with the feedback._user, _organisation, _prompt and _environment are special fields indexed by default	string

Examples

A simple & effective feedback from the user is a thumbs up or thumbs down. Just set value to 1 for 👍 and -1 for 👎. Weight would be default 1.0.

cURL

curl --location 'https://aigw.portkey.ai/v1/feedback' \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data '{
    'trace_id': 'REQUEST_TRACE_ID',
    'value': 1
}'

Other Ideas for collecting feedback

Business metrics make for great feedback. If you’re generating an email, the email being sent out could be a positive feedback metric. The level of editing could indicate the value.
When a user retries a generation, store the negative feedback since something probably went wrong. Use a lower weight for this feedback since it could be circumstantial.

Feedback Analytics

You can see the Feedback Count and Value: Weight pairs for each trace-id on the logs page. You can also view the feedback details on Analytics and on the Prompt Eval tabs.

Introduction

Product

Adding Feedback to Requests

1. Find the `trace-id`

2. Add feedback

Examples

Other Ideas for collecting feedback

Feedback Analytics

​Adding Feedback to Requests

​1. Find the `trace-id`

​2. Add feedback

​Examples

​Other Ideas for collecting feedback

​Feedback Analytics

Adding Feedback to Requests

1. Find the `trace-id`

2. Add feedback

Examples

Other Ideas for collecting feedback

Feedback Analytics