Building production-grade chatbots comes with its share of challenges. From managing conversation context and engineering prompts to ensuring consistent responses across different scenarios – the development process can quickly become complex. Add in the need for version control, testing different configurations, and maintaining production stability, and you’ve got quite a puzzle to solve. This is where Portkey’s prompt templates come in. More than just a development tool, Portkey provides a complete ecosystem for building, testing, and deploying chatbots with confidence. You’ll be able to:

  • Experiment with different prompt configurations while maintaining version control
  • Test your chatbot’s responses in real-time with an interactive playground
  • Portkey’s robust versioning system ensures that you can experiment freely with your prompts, allowing for easy rollback.
  • Experiment with different models and configurations to find the best fit for your use case

In this guide, we’ll walk through the process of building a production-ready chatbot using Portkey’s prompt templates. Whether you’re creating a customer service bot, a knowledge assistant, or any other conversational AI application, you’ll learn how to leverage Portkey’s features to build a robust solution that scales. Here’s the link to the collab notebook of the chatbot-

​
Setting Up Your Chatbot

Go to Portkey’s Prompts dashboard. Click on the Create button. You are now on Prompt Playground.

​
Step 1: Define Your System Prompt

Start by defining your system prompt. This sets the initial context and behavior for your chatbot. You can set this up in your Portkey’s Prompt Library using the JSON View

[
  {
    "content": "You're a helpful assistant.",
    "role": "system"
  },
  {{chat_history}}
]

​
Step 2: Create a Variable to Store Conversation History

In the Portkey UI, set the variable type: Look for two icons next to the variable name: “T” and ”{..}”. Click the ”{…}” icon to switch to JSON mode.

Initialize the variable: This array will store the conversation history, allowing your chatbot to maintain context. We can just initialize the variable with [].

As your chatbot interacts with users, it will append new messages to this array, building a comprehensive conversation history.

​
Step 3: Implementing the Chatbot

Use Portkey’s API to generate responses based on your prompt template. Here’s a Python example::

from portkey_ai import Portkey
client = Portkey(
    api_key="YOUR_PORTKEY_API_KEY"  # You can also set this as an environment variable
)

def generate_response(conversation_history):
    prompt_completion = client.prompts.completions.create(
        prompt_id="YOUR_PROMPT_ID",  # Replace with your actual prompt ID
        variables={
            "variable": conversation_history
        }
    )

    return prompt_completion.choices[0].message.content

# Example usage

conversation_history = [
    {
        "content": "Hello, how can I assist you today?",
        "role": "assistant"
    },
    {
        "content": "What's the weather like?",
        "role": "user"
    }
]

response = generate_response(conversation_history)

print(response)

​
Step 4: Append the Response

After generating a response, append it to your conversation history:

def append_response(conversation_history, response):

    conversation_history.append({

        "content": response,

        "role": "assistant"

    })

    return conversation_history

# Continuing from the previous example

conversation_history = append_response(conversation_history, response)

​
Step 5: Take User Input to Continue the Conversation

Implement a loop to continuously take user input and generate responses:

# Continue the conversation
while True:
    user_input = input("You: ")
    if user_input.lower() == 'exit':
        break

    conversation_history.append({
        "content": user_input,
        "role": "user"
    })

    response = generate_response(conversation_history)
    conversation_history = append_response(conversation_history, response)

    print("Bot:", response)
print("Conversation ended.")

​
Complete Example

Here’s a complete example that puts all these steps together:

from portkey_ai import Portkey

client = Portkey(
    api_key="YOUR_PORTKEY_API_KEY"
)

def generate_response(conversation_history):
    prompt_completion = client.prompts.completions.create(
        prompt_id="YOUR_PROMPT_ID",
        variables={
            "variable": conversation_history
        }
    )

    return prompt_completion.choices[0].message.content

def append_response(conversation_history, response):
    conversation_history.append({
        "content": response,
        "role": "assistant"
    })

    return conversation_history

# Initial conversation

conversation_history = [
    {
        "content": "Hello, how can I assist you today?",
        "role": "assistant"
    }
]

# Generate and append response

response = generate_response(conversation_history)

conversation_history = append_response(conversation_history, response)

print("Bot:", response)

# Continue the conversation

while True:
    user_input = input("You: ")
    if user_input.lower() == 'exit':
        break

    conversation_history.append({
        "content": user_input,
        "role": "user"
    })

    response = generate_response(conversation_history)
    conversation_history = append_response(conversation_history, response)

    print("Bot:", response)

print("Conversation ended.")

​
Conclusion

Voilà! You’ve successfully set up your chatbot using Portkey’s prompt templates. Portkey enables you to experiment with various LLM providers. It acts as a definitive source of truth for your team, and it versions each snapshot of model parameters, allowing for easy rollback. Here’s a snapshot of the Prompt Management UI. To learn more about Prompt Management click here.

Was this page helpful?

Suggest editsRaise issue
Comparing Top10 LMSYS Models with PortkeyTracking LLM Costs Per User with Portkey