> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deepinfra

> Integrate Deepinfra models with Prisma AIRS AI Gateway

The AI Gateway provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including [Deepinfra's hosted models](https://deepinfra.com/models/text-generation).

With the AI Gateway, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through [Model Catalog](/docs/aigw/product/model-catalog).

## Quick Start

Get Deepinfra working in 3 steps:

<CodeGroup>
  ```sh cURL icon="square-terminal" theme={"system"}
  # 1. Add @deepinfra provider in model catalog
  # 2. Use it:

  curl https://aigw.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $PORTKEY_API_KEY" \
    -d '{
      "model": "@deepinfra/nvidia/Nemotron-4-340B-Instruct",
      "messages": [
        { "role": "user", "content": "Say this is a test" }
      ]
    }'
  ```

  ```python OpenAI Py icon="python" theme={"system"}
  from openai import OpenAI

  # 1. Install: pip install openai
  # 2. Add @deepinfra provider in model catalog
  # 3. Use it:

  client = OpenAI(
      api_key="PORTKEY_API_KEY",  # AI Gateway API key
      base_url="https://aigw.portkey.ai/v1"
  )

  response = client.chat.completions.create(
      model="@deepinfra/nvidia/Nemotron-4-340B-Instruct",
      messages=[{"role": "user", "content": "Say this is a test"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js OpenAI JS icon="square-js" theme={"system"}
  import OpenAI from "openai"

  // 1. Install: npm install openai
  // 2. Add @deepinfra provider in model catalog
  // 3. Use it:

  const client = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // AI Gateway API key
      baseURL: "https://aigw.portkey.ai/v1"
  })

  const response = await client.chat.completions.create({
      model: "@deepinfra/nvidia/Nemotron-4-340B-Instruct",
      messages: [{ role: "user", content: "Say this is a test" }]
  })

  console.log(response.choices[0].message.content)
  ```
</CodeGroup>

<Note>
  **Tip:** You can also send `x-portkey-provider: @deepinfra` as a header and use just `model="nvidia/Nemotron-4-340B-Instruct"` in the request.
</Note>

## Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://stratacloudmanager.paloaltonetworks.com/)
2. Select **Deepinfra**
3. Choose existing credentials or create new by entering your [Deepinfra API key](https://deepinfra.com/dash/api_keys)
4. Name your provider (e.g., `deepinfra-prod`)

<Card title="Complete Setup Guide →" href="/docs/aigw/product/model-catalog">
  See all setup options, code examples, and detailed instructions
</Card>

## Supported Endpoints

| Endpoint            | Supported |
| ------------------- | --------- |
| `/chat/completions` | ✅         |
| `/completions`      | ✅         |
| `/embeddings`       | ✅         |

## Tool Calling

DeepInfra supports tool calling (function calling) for compatible models. Use the standard OpenAI `tools` format:

## Supported Models

Deepinfra hosts a wide range of open-source models for text generation. View the complete list:

<Card title="Deepinfra Models" icon="list" href="https://deepinfra.com/models/text-generation">
  Browse all available models on Deepinfra
</Card>

Popular models include:

* `nvidia/Nemotron-4-340B-Instruct`
* `meta-llama/Meta-Llama-3.1-405B-Instruct`
* `Qwen/Qwen2.5-72B-Instruct`

## Next Steps

<CardGroup cols={2}>
  <Card title="Add Metadata" icon="tags" href="/docs/aigw/product/observability/metadata">
    Add metadata to your Deepinfra requests
  </Card>

  <Card title="Gateway Configs" icon="gear" href="/docs/aigw/product/ai-gateway/configs">
    Add gateway configs to your Deepinfra requests
  </Card>

  <Card title="Tracing" icon="chart-line" href="/docs/aigw/product/observability/traces">
    Trace your Deepinfra requests
  </Card>

  <Card title="Fallbacks" icon="arrow-rotate-left" href="/docs/aigw/product/ai-gateway/fallbacks">
    Setup fallback from OpenAI to Deepinfra
  </Card>
</CardGroup>


## Related topics

- [Enterprise Gateway](/docs/aigw/changelog/enterprise.md)
- [Prisma AIRS AI Gateway Models](/docs/aigw/product/model-catalog/gateway-models.md)
- [Overview](/docs/aigw/integrations/llms.md)
- [Supported Providers](/docs/aigw/api-reference/inference-api/supported-providers.md)