Function Calling
Portkey’s AI Gateway supports function calling capabilities that many foundational model providers offer. In the API call you can describe functions and the model can choose to output text or this function name with parameters.
Functions Usage
Portkey supports the OpenAI signature to define functions as part of the API request. The tools
parameter accepts functions which can be sent specifically for models that support function/tool calling.
API Reference
On completion, the request will get logged in the logs UI where the tools and functions can be viewed. Portkey will automatically format the JSON blocks in the input and output which makes a great debugging experience.
Managing Functions and Tools in Prompts
Portkey’s Prompt Library supports creating prompt templates with function/tool definitions, as well as letting you set the tool choice
param. Portkey will also validate your tool definition on the fly, eliminating syntax errors.
Supported Providers and Models
The following providers are supported for function calling with more providers getting added soon. Please raise a request or a PR to add model or provider to the AI gateway.
Provider | Models |
---|---|
OpenAI | gpt-4 series of modelsgpt-3.5-turbo series of models |
Azure OpenAI | gpt-4 series of modelsgpt-3.5-turbo series of models |
Anyscale | mistralai/Mistral-7B-Instruct-v0.1 mistralai/Mixtral-8x7B-Instruct-v0.1 |
Together AI | mistralai/Mixtral-8x7B-Instruct-v0.1 mistralai/Mistral-7B-Instruct-v0.1 togethercomputer/CodeLlama-34b-Instruct |
Fireworks AI | firefunction-v1fw-function-call-34b-v0 |
Google Gemini / Vertex AI | gemini-1.0-progemini-1.0-pro-001gemini-1.5-pro-latest |
Cookbook
Here’s a detailed cookbook on function calling using Portkey.
Was this page helpful?