OpenAI Swarm

Swarm is an experimental framework by OpenAI for building multi-agent systems. It showcases the handoff & routines pattern, making agent coordination and execution lightweight, highly controllable, and easily testable. Portkey integration extends Swarm’s capabilities with production-ready features like observability, reliability, and more.

​

Getting Started

pip install -U portkey-ai

from swarm import Swarm, Agent
from portkey_ai import Portkey

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY", # defaults to os.environ.get("PORTKEY_API_KEY")
    virtual_key="YOUR_VIRTUAL_KEY" 
    )

client = Swarm(client=portkey)

def get_weather(location) -> str:
    return "{'temp':67, 'unit':'F'}"


agent = Agent(
    name="Agent",
    instructions="You are a helpful agent.",
    functions=[get_weather],
)

messages = [{"role": "user", "content": "What's the weather in NYC?"}]

response = client.run(agent=agent, messages=messages)
print(response.messages[-1]["content"])

​

E2E example with Function Calling in OpenAI Swarm

Here’s a complete example showing function calling and agent interaction:

from swarm import Swarm, Agent
from portkey_ai import Portkey

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY", # defaults to os.environ.get("PORTKEY_API_KEY")
    virtual_key="YOUR_VIRTUAL_KEY" 
    )

client = Swarm(client=portkey)


def get_weather(location) -> str:
    return "{'temp':67, 'unit':'F'}"


agent = Agent(
    name="Agent",
    instructions="You are a helpful agent.",
    functions=[get_weather],
)

messages = [{"role": "user", "content": "What's the weather in NYC?"}]

response = client.run(agent=agent, messages=messages)
print(response.messages[-1]["content"])

The current temperature in New York City is 67°F.

​

Enabling Portkey Features

By routing your OpenAI Swarm requests through Portkey, you get access to the following production-grade features:

​

1. Interoperability - Calling Different LLMs

When building with Swarm, you might want to experiment with different LLMs or use specific providers for different agent tasks. Portkey makes this seamless - you can switch between OpenAI, Anthropic, Gemini, Mistral, or cloud providers without changing your agent code.

Instead of managing multiple API keys and provider-specific configurations, Portkey’s Virtual Keys give you a single point of control. Here’s how you can use different LLMs with your Swarm agents:

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY",
    virtual_key="ANTHROPIC_VIRTUAL_KEY" #Just change the virtual key to your preferred LLM provider
)

client = Swarm(client=portkey)

​

2. Caching - Speed Up Agent Responses

Agent operations often involve repetitive queries or similar tasks. Every time your agent makes an LLM call, you’re paying for tokens and waiting for responses. Portkey’s caching system can significantly reduce both costs and latency.

Portkey offers two powerful caching modes:

Simple Cache: Perfect for exact matches - when your agents make identical requests. Ideal for deterministic operations like function calling or FAQ-type queries.

Semantic Cache: Uses embedding-based matching to identify similar queries. Great for natural language interactions where users might ask the same thing in different ways.

config = {
    "cache": {
        "mode": "semantic",  # or "simple" for exact matching
        "max_age": 3600000  # cache duration in milliseconds
    }
}

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY",
    virtual_key="YOUR_VIRTUAL_KEY",
    config=config
)

​

3. Reliability - Keep Your Agents Running Smoothly

When running agents in production, things can go wrong - API rate limits, network issues, or provider outages. Portkey’s reliability features ensure your agents keep running smoothly even when problems occur.

​

4. Observability - Understand Your Agents

Building agents is the first step - but how do you know they’re working effectively? Portkey provides comprehensive visibility into your agent operations through multiple lenses:

Metrics Dashboard: Track 40+ key performance indicators like:

• Cost per agent interaction
• Response times and latency
• Token usage and efficiency
• Success/failure rates
• Cache hit rates

​

Send Custom Metadata with your requests

Add trace IDs to track specific workflows:

portkey = Portkey(
    api_key="YOUR_PORTKEY_API_KEY",
    virtual_key="YOUR_VIRTUAL_KEY",
    trace_id="weather_workflow_123",
    metadata={
        "agent": "weather_agent",
        "environment": "production"
    }
)

​

5. Logs and Traces

Logs are essential for understanding agent behavior, diagnosing issues, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes.

Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata.

​

6. Security & Compliance - Enterprise-Ready Controls

When deploying agents in production, security is crucial. Portkey provides enterprise-grade security features:

Configure these settings in the Portkey Dashboard or programmatically through the API.

​

7. Continuous Improvement

Now that you know how to trace & log your Llamaindex requests to Portkey, you can also start capturing user feedback to improve your app!

You can append qualitative as well as quantitative feedback to any trace ID with the portkey.feedback.create method:

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    virtual_key="YOUR_OPENAI_VIRTUAL_KEY"
)

feedback = portkey.feedback.create(
    trace_id="YOUR_LLAMAINDEX_TRACE_ID",
    value=5,  # Integer between -10 and 10
    weight=1,  # Optional
    metadata={
        # Pass any additional context here like comments, _user and more
    }
)

print(feedback)

​

Portkey Config

Many of these features are driven by Portkey’s Config architecture. The Portkey app simplifies creating, managing, and versioning your Configs.

For more information on using these features and setting up your Config, please refer to the Portkey documentation.