> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Together AI

> Integrate Together AI models with Prisma AIRS AI Gateway

The AI Gateway provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including [Together AI's hosted models](https://docs.together.ai/reference/inference).

With the AI Gateway, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through [Model Catalog](/docs/aigw/product/model-catalog).

## Quick Start

Get Together AI working in 3 steps:

<CodeGroup>
  ```sh cURL icon="square-terminal" theme={"system"}
  # 1. Add @together-ai provider in model catalog
  # 2. Use it:

  curl https://aigw.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PORTKEY_API_KEY" \
    -d '{
      "model": "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      "messages": [
        { "role": "user", "content": "Say this is a test" }
      ]
    }'
  ```

  ```python OpenAI Py icon="python" theme={"system"}
  from openai import OpenAI

  # 1. Install: pip install openai
  # 2. Add @together-ai provider in model catalog
  # 3. Use it:

  client = OpenAI(
      api_key="PORTKEY_API_KEY",  # AI Gateway API key
      base_url="https://aigw.portkey.ai/v1"
  )

  response = client.chat.completions.create(
      model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      messages=[{"role": "user", "content": "Say this is a test"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js OpenAI JS icon="square-js" theme={"system"}
  import OpenAI from "openai"

  // 1. Install: npm install openai
  // 2. Add @together-ai provider in model catalog
  // 3. Use it:

  const client = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // AI Gateway API key
      baseURL: "https://aigw.portkey.ai/v1"
  })

  const response = await client.chat.completions.create({
      model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      messages: [{ role: "user", content: "Say this is a test" }]
  })

  console.log(response.choices[0].message.content)
  ```
</CodeGroup>

<Note>
  **Tip:** You can also send `x-portkey-provider: @together-ai` as a header and use just `model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"` in the request.
</Note>

## Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
2. Select **Together AI**
3. Choose existing credentials or create new by entering your [Together AI API key](https://api.together.ai/settings/api-keys)
4. Name your provider (e.g., `together-ai-prod`)

<Card title="Complete Setup Guide →" href="/docs/aigw/product/model-catalog">
  See all setup options, code examples, and detailed instructions
</Card>

## Reasoning / Thinking Support

Together AI supports reasoning models that expose their internal chain of thought. Use the `reasoning_effort` parameter to control reasoning behavior, and set `strict_open_ai_compliance=False` to receive the thinking content in `content_blocks`.

<CodeGroup>
  ```sh cURL theme={"system"}
  curl https://aigw.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PORTKEY_API_KEY" \
    -H "x-portkey-strict-open-ai-compliance: false" \
    -d '{
      "model": "@together-ai/deepseek-ai/DeepSeek-R1",
      "messages": [
        { "role": "user", "content": "Solve step by step: What is 23 * 47?" }
      ],
      "reasoning_effort": "medium"
    }'
  ```
</CodeGroup>

The reasoning response includes `content_blocks` with both the model's thinking process and the final answer. Streaming is also supported and returns reasoning chunks in the `content_blocks` field of the stream delta.

<Card title="Thinking Mode Documentation" icon="brain" href="/docs/aigw/product/ai-gateway/multimodal-capabilities/thinking-mode">
  Learn more about thinking/reasoning support across providers
</Card>

## Video Generation

Together AI supports video generation through the AI Gateway's proxy. Generate videos from text prompts and track usage in Strata Cloud Manager.

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/v2/videos' \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer PORTKEY_API_KEY' \
    --data '{
      "prompt": "A serene sunset over the ocean with gentle waves",
      "model": "@together-ai/minimax/hailuo-02",
      "width": 1366,
      "height": 768
    }'
  ```
</CodeGroup>

Video generation pricing is logged in Strata Cloud Manager for cost tracking.

***

## Managing Together AI Prompts

Manage all prompt templates to Together AI in the Prompt Library. All current Together AI models are supported, and you can easily test different prompts.

Call the `POST /v1/prompts/{promptId}/completions` endpoint to use the prompt in an application.

## Next Steps

<CardGroup cols={2}>
  <Card title="Add Metadata" icon="tags" href="/docs/aigw/product/observability/metadata">
    Add metadata to your Together AI requests
  </Card>

  <Card title="Gateway Configs" icon="gear" href="/docs/aigw/product/ai-gateway/configs">
    Add gateway configs to your Together AI requests
  </Card>

  <Card title="Tracing" icon="chart-line" href="/docs/aigw/product/observability/traces">
    Trace your Together AI requests
  </Card>

  <Card title="Fallbacks" icon="arrow-rotate-left" href="/docs/aigw/product/ai-gateway/fallbacks">
    Setup fallback from OpenAI to Together AI
  </Card>
</CardGroup>


## Related topics

- [Enterprise Gateway](/docs/aigw/changelog/enterprise.md)
- [Thinking Mode](/docs/aigw/product/ai-gateway/multimodal-capabilities/thinking-mode.md)
- [Prisma AIRS AI Gateway Models](/docs/aigw/product/model-catalog/gateway-models.md)
- [Overview](/docs/aigw/integrations/llms.md)
- [Messages](/docs/aigw/product/ai-gateway/messages-api.md)