> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Vision

> The Prisma AIRS AI Gateway's AI gateway supports vision models like GPT-4V by OpenAI, Gemini by Google and more.

<Info>
  **What are vision models?**

  Vision models are artificial intelligence systems that combine both vision and language modalities to process images and natural language text. These models are typically trained on large image and text datasets with different structures based on the pre-training objective.
</Info>

## Vision Chat Completion Usage

The AI Gateway supports the OpenAI signature to define messages with images as part of the API request. Images are made available to the model in two main ways: by passing a link to the image or by passing the base64 encoded image directly in the request.

Here's an example using OpenAI's `gpt-4o` model

<Tabs>
  <Tab title="OpenAI NodeJS">
    ```js theme={"system"}
    import OpenAI from 'openai'; // We're using the v4 SDK

    const openai = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // defaults to process.env["PORTKEY_API_KEY"]
      baseURL: "https://aigw.portkey.ai/v1",
      defaultHeaders: {
      }
    });

    // Generate a chat completion with streaming
    async function getChatCompletionFunctions(){
      const response = await openai.chat.completions.create({
          model: "@openai/gpt-4o-mini",
          messages: [{
              role: "user",
              content: [
                  { type: "text", text: "What is in this image?" },
                  {
                      type: "image_url",
                      image_url: {
                          url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
                      },
                  },
              ],
          }],
      });

      console.log(response)

    }
    await getChatCompletionFunctions();
    ```
  </Tab>

  <Tab title="OpenAI Python">
    ```py theme={"system"}
    from openai import OpenAI

    openai = OpenAI(
        api_key="PORTKEY_API_KEY",
        base_url="https://aigw.portkey.ai/v1",
    )

    response = openai.chat.completions.create(
        model="@openai/gpt-4o-mini",
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
                    },
                },
            ],
        }],
    )

    print(resonse)
    ```
  </Tab>

  <Tab title="cURL">
    ```sh theme={"system"}
    curl "https://aigw.portkey.ai/v1/chat/completions" \
      -H "Content-Type: application/json" \
      -H "x-portkey-api-key: $PORTKEY_API_KEY" \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -d '{
          "model": "@openai/gpt-4o-mini",
          "messages": [
            {
              "role": "user",
              "content": [
                {
                  "type": "text",
                  "text": "What is in this image?"
                },
                {
                  "type": "image_url",
                  "image_url": {
                    "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
                  }
                }
              ]
            }
          ],
          "max_tokens": 300
        }'
    ```
  </Tab>
</Tabs>

### [API Reference](/docs/aigw/product/ai-gateway/multimodal-capabilities/vision#vision-chat-completion-usage)

On completion, the request will get logged in the logs UI where any image inputs or outputs can be viewed. The AI Gateway will automatically load the image URLs or the base64 images making for a great debugging experience with vision models.

## Creating prompt templates for vision models

The AI Gateway's prompt library supports creating templates with image inputs. If the same image will be used in all prompt calls, you can save it as part of the template's image URL itself. Or, if the image will be sent via the API as a variable, add a variable to the image link.

## Supported Providers and Models

The AI Gateway supports all vision models from its integrated providers as they become available. The table below shows some examples of supported vision models. Please raise a [request](/docs/aigw/integrations/llms/suggest-a-new-integration) to add a provider to the AI gateway.

| Provider                                        | Models                                                                                             | Functions              |
| ----------------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------- |
| [OpenAI](/docs/aigw/integrations/llms/openai)        | `gpt-4-vision-preview`, `gpt-4o`, `gpt-4o-mini `                                                   | Create Chat Completion |
| [Azure OpenAI](/docs/integrations/llms/azure-openai) | `gpt-4-vision-preview`, `gpt-4o`, `gpt-4o-mini `                                                   | Create Chat Completion |
| [Gemini](/docs/aigw/integrations/llms/gemini)        | `gemini-1.0-pro-vision `, `gemini-1.5-flash`, `gemini-1.5-flash-8b`, `gemini-1.5-pro`              | Create Chat Completion |
| [Anthropic](/docs/aigw/integrations/llms/anthropic)  | `claude-3-sonnet`, `claude-3-haiku`, `claude-3-opus`,  `claude-3.5-sonnet`, `claude-3.5-haiku`     | Create Chat Completion |
| [AWS Bedrock](/docs/integrations/llms/aws-bedrock)   | `anthropic.claude-3-5-sonnet anthropic.claude-3-5-haiku anthropic.claude-3-5-sonnet-20240620-v1:0` | Create Chat Completion |

For a complete list of all supported provider (including non-vision LLMs), check out our [providers documentation](/docs/aigw/integrations/llms).


## Related topics

- [xAI (Grok)](/docs/aigw/integrations/llms/x-ai.md)
- [Chat Completions](/docs/aigw/product/ai-gateway/chat-completions.md)
- [Messages](/docs/aigw/product/ai-gateway/messages-api.md)
- [Open Responses](/docs/aigw/product/ai-gateway/responses-api.md)
- [OpenAI](/docs/aigw/integrations/llms/openai.md)