Instructor is a framework for extracting structured outputs from LLMs, available in Python & JS.

​
Integrating Portkey with Instructor

import instructor
from pydantic import BaseModel
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

portkey = OpenAI(
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        virtual_key="OPENAI_VIRTUAL_KEY",
        api_key="PORTKEY_API_KEY"
    )
)

class User(BaseModel):
    name: str
    age: int

client = instructor.from_openai(portkey)

user_info = client.chat.completions.create(
    model="gpt-4-turbo",
    max_tokens=1024,
    response_model=User,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)

print(user_info.name)
print(user_info.age)

​
Caching Your Requests

Let’s now bring down the cost of running your Instructor pipeline with Portkey caching. You can just create a Config object where you define your cache setting:

{
  "cache": {
    "mode": "simple"
  }
}

You can write it raw, or use Portkey’s Config builder and get a corresponding config id. Then, just pass it while instantiating your OpenAI client:

import instructor
from pydantic import BaseModel
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

cache_config = {
  "cache": {
    "mode": "simple"
  }
}

portkey = OpenAI(
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        virtual_key="OPENAI_VIRTUAL_KEY",
        api_key="PORTKEY_API_KEY",
        config=cache_config # Or pass your Config ID saved from Portkey app
    )
)

class User(BaseModel):
    name: str
    age: int

client = instructor.from_openai(portkey)

user_info = client.chat.completions.create(
    model="gpt-4-turbo",
    max_tokens=1024,
    response_model=User,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)

print(user_info.name)
print(user_info.age)

Similarly, you can add Fallback, Loadbalancing, Timeout, or Retry settings to your Configs and make your Instructor requests robust & reliable.

Was this page helpful?

DSPyLangchain (Python)