Skip to main content
The AI Gateway provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Together AI’s hosted models. With the AI Gateway, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

Quick Start

Get Together AI working in 3 steps:
Tip: You can also send x-portkey-provider: @together-ai as a header and use just model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" in the request.

Add Provider in Model Catalog

  1. Go to Model Catalog → Add Provider
  2. Select Together AI
  3. Choose existing credentials or create new by entering your Together AI API key
  4. Name your provider (e.g., together-ai-prod)

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

Reasoning / Thinking Support

Together AI supports reasoning models that expose their internal chain of thought. Use the reasoning_effort parameter to control reasoning behavior, and set strict_open_ai_compliance=False to receive the thinking content in content_blocks.
The reasoning response includes content_blocks with both the model’s thinking process and the final answer. Streaming is also supported and returns reasoning chunks in the content_blocks field of the stream delta.

Thinking Mode Documentation

Learn more about thinking/reasoning support across providers

Video Generation

Together AI supports video generation through the AI Gateway’s proxy. Generate videos from text prompts and track usage in Strata Cloud Manager.
Video generation pricing is logged in Strata Cloud Manager for cost tracking.

Managing Together AI Prompts

Manage all prompt templates to Together AI in the Prompt Library. All current Together AI models are supported, and you can easily test different prompts. Call the POST /v1/prompts/{promptId}/completions endpoint to use the prompt in an application.

Next Steps

Add Metadata

Add metadata to your Together AI requests

Gateway Configs

Add gateway configs to your Together AI requests

Tracing

Trace your Together AI requests

Fallbacks

Setup fallback from OpenAI to Together AI
Last modified on September 15, 2026