> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Embeddings

> Get embeddings from Vertex AI

Vertex AI offers wide ranging support for embedding text, images and videos.
The Prisma AIRS AI Gateway provides a standardized interface for embedding multiple modalities.

***

## Gemini Embedding Models

The `gemini-embedding-2-preview` model supports embedding across multiple modalities — **text**, **image**, **video**, and **audio** — through a single unified endpoint.

| Input Type | Supported Formats                    |
| ---------- | ------------------------------------ |
| Text       | Plain string or structured object    |
| Image      | GCS URI, HTTPS URL, base64, data URI |
| Video      | GCS URI, HTTPS URL, base64           |
| Audio      | GCS URI, HTTPS URL, base64           |

Additional supported parameters: `task_type`, `dimensions`

### Embedding Text

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-vertex-region: us-central1' \
  --data '{
      "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      "input": "What is the meaning of life?"
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-vertex-region": "us-central1"}
  )

  embeddings = gateway_client.embeddings.create(
      model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      input="What is the meaning of life?",
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai';

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai/v1",
    defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
  });

  const embedding = await gatewayClient.embeddings.create({
      input: "What is the meaning of life?",
      model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
  });

  console.log(embedding);
  ```
</CodeGroup>

### Embedding Images

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-vertex-region: us-central1' \
  --data '{
      "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      "input": [
          {
              "image": {
                  "url": "gs://your-bucket/image.png",
                  "mime_type": "image/png"
              }
          }
      ]
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-vertex-region": "us-central1"}
  )

  embeddings = gateway_client.embeddings.create(
      model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      input=[
          {
              "image": {
                  "url": "gs://your-bucket/image.png",
                  "mime_type": "image/png"
              }
          }
      ],
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai';

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai/v1",
    defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
  });

  const embedding = await gatewayClient.embeddings.create({
      input: [
          {
              image: {
                  url: "gs://your-bucket/image.png",
                  mime_type: "image/png"
              }
          }
      ],
      model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
  });

  console.log(embedding);
  ```
</CodeGroup>

You can also pass images as base64:

```json theme={"system"}
{
    "input": [
        {
            "image": {
                "base64": "iVBORw0KGgoAAAANSUhEUgAA...",
                "mime_type": "image/png"
            }
        }
    ]
}
```

Or as a data URI:

```json theme={"system"}
{
    "input": [
        {
            "image": {
                "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
            }
        }
    ]
}
```

### Embedding Videos

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-vertex-region: us-central1' \
  --data '{
      "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      "input": [
          {
              "video": {
                  "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                  "mime_type": "video/mp4"
              }
          }
      ]
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-vertex-region": "us-central1"}
  )

  embeddings = gateway_client.embeddings.create(
      model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      input=[
          {
              "video": {
                  "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                  "mime_type": "video/mp4"
              }
          }
      ],
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai';

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai/v1",
    defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
  });

  const embedding = await gatewayClient.embeddings.create({
      input: [
          {
              video: {
                  url: "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                  mime_type: "video/mp4"
              }
          }
      ],
      model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
  });

  console.log(embedding);
  ```
</CodeGroup>

### Embedding Audio

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-vertex-region: us-central1' \
  --data '{
      "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      "input": [
          {
              "audio": {
                  "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                  "mime_type": "audio/mpeg"
              }
          }
      ]
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-vertex-region": "us-central1"}
  )

  embeddings = gateway_client.embeddings.create(
      model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      input=[
          {
              "audio": {
                  "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                  "mime_type": "audio/mpeg"
              }
          }
      ],
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai';

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai/v1",
    defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
  });

  const embedding = await gatewayClient.embeddings.create({
      input: [
          {
              audio: {
                  url: "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                  mime_type: "audio/mpeg"
              }
          }
      ],
      model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
  });

  console.log(embedding);
  ```
</CodeGroup>

### Multimodal Embedding (Mixed Inputs)

You can combine multiple input types in a single request:

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-vertex-region: us-central1' \
  --data '{
      "model": "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      "input": [
          {
              "video": {
                  "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                  "mime_type": "video/mp4"
              }
          },
          {
              "audio": {
                  "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                  "mime_type": "audio/mpeg"
              }
          }
      ]
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-vertex-region": "us-central1"}
  )

  embeddings = gateway_client.embeddings.create(
      model="@PORTKEY_PROVIDER/gemini-embedding-2-preview",
      input=[
          {
              "video": {
                  "url": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                  "mime_type": "video/mp4"
              }
          },
          {
              "audio": {
                  "url": "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                  "mime_type": "audio/mpeg"
              }
          }
      ],
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai';

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY",
    baseURL: "https://aigw.portkey.ai/v1",
    defaultHeaders: { "x-portkey-vertex-region": "us-central1" }
  });

  const embedding = await gatewayClient.embeddings.create({
      input: [
          {
              video: {
                  url: "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
                  mime_type: "video/mp4"
              }
          },
          {
              audio: {
                  url: "gs://cloud-samples-data/generative-ai/audio/Chirp-3-Docs-Dive.mp3",
                  mime_type: "audio/mpeg"
              }
          }
      ],
      model: "@PORTKEY_PROVIDER/gemini-embedding-2-preview",
  });

  console.log(embedding);
  ```
</CodeGroup>

### Setting Task Type and Dimensions

You can optionally specify `task_type` and `dimensions` to control the embedding behavior:

```json theme={"system"}
{
    "model": "gemini-embedding-2-preview",
    "input": "What is the meaning of life?",
    "task_type": "RETRIEVAL_DOCUMENT",
    "dimensions": 768
}
```

***

## Legacy Embedding Models

The following sections cover the older Vertex AI embedding models like `textembedding-gecko@003` and `multimodalembedding@001`.

## Embedding Text

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-provider: PORTKEY_PROVIDER' \
  --data-raw '{
      "model": "textembedding-gecko@003",
      "input": [
          "A HTTP 246 code is used to signify an AI response containing hallucinations or other inaccuracies",
          "246: Partially incorrect response"
      ],
      # "input": "Name the tallest buildings in Hawaii",
      "input_type": "classification"
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-provider": "openai"}
  )

  embeddings = gateway_client.embeddings.create(
    model="textembedding-gecko@003",
    input_type="classification",
    input="The food was delicious and the waiter...",
    # input=["text to embed", "more text to embed"], # if you would like to embed multiple texts
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai'; // We're using the v4 SDK

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
    baseURL: "https://aigw.portkey.ai/v1"
  });

  const embedding = await gatewayClient.embeddings.create({
      input: 'Name the tallest buildings in Hawaii',
      // input: ['text to embed', 'more text to embed'], // if you would like to embed multiple texts
      model: '@PORTKEY_PROVIDER/textembedding-gecko@003'
  });

  console.log(embedding);
  ```
</CodeGroup>

## Embeddings Images

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-provider: PORTKEY_PROVIDER' \
  --data-raw '{
      "model": "multimodalembedding@001",
      "input": [
                    {
                        "text": "this is the caption of the image",
                        "image": {
                            "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B....."
                            # "url": "gcs://..." # if you want to use a url
                        }
                    }
                ]
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-provider": "openai"}
  )

  embeddings = gateway_client.embeddings.create(
    model="multimodalembedding@001",
    input=[
            {
                "text": "this is the caption of the image",
                "image": {
                    "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                    # "url": "gcs://..." # if you want to use a url
                }
            }
          ]
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai'; // We're using the v4 SDK

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
    baseURL: "https://aigw.portkey.ai/v1"
  });

  const embedding = await gatewayClient.embeddings.create({
      input: [
                {
                    "text": "this is the caption of the image",
                    "image": {
                        "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                        // "url": "gcs://..." // if you want to use a url
                    }
                }
              ],
      model: '@PORTKEY_PROVIDER/multimodalembedding@001'
  });

  console.log(embedding);
  ```
</CodeGroup>

## Embeddings Videos

<CodeGroup>
  ```sh cURL theme={"system"}
  curl --location 'https://aigw.portkey.ai/v1/embeddings' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer PORTKEY_API_KEY' \
  --header 'x-portkey-provider: PORTKEY_PROVIDER' \
  --data-raw '{
      "model": "multimodalembedding@001",
      "input": [
                    {
                        "text": "this is the caption of the video",
                        "video": {
                            "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                            "start_offset": 0,
                            "end_offset": 10,
                            "interval": 5
                            # "url": "gcs://..." # if you want to use a url
                        }
                    }
                ]
  }'
  ```

  ```python OpenAI Python theme={"system"}
  from openai import OpenAI

  gateway_client = OpenAI(
      api_key="PORTKEY_API_KEY",
      base_url="https://aigw.portkey.ai/v1",
      default_headers={"x-portkey-provider": "openai"}
  )

  embeddings = gateway_client.embeddings.create(
    model="multimodalembedding@001",
    input=[
            {
                "text": "this is the caption of the video",
                "video": {
                    "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                    "start_offset": 0,
                    "end_offset": 10,
                    "interval": 5,
                    # "url": "gcs://..." # if you want to use a url
                }
            }
          ]
  )
  ```

  ```js OpenAI NodeJS theme={"system"}
  import OpenAI from 'openai'; // We're using the v4 SDK

  const gatewayClient = new OpenAI({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["OPENAI_API_KEY"],
    baseURL: "https://aigw.portkey.ai/v1"
  });

  const embedding = await gatewayClient.embeddings.create({
      input: [
                {
                    "text": "this is the caption of the video",
                    "video": {
                        "base64": "UklGRkacAABXRUJQVlA4IDqcAACQggKdASqpAn8B.....",
                        "start_offset": 0,
                        "end_offset": 10,
                        "interval": 5,
                        // "url": "gcs://..." // if you want to use a url
                    }
                }
              ],
      model: '@PORTKEY_PROVIDER/multimodalembedding@001'
  });

  console.log(embedding);
  ```
</CodeGroup>


## Related topics

- [Embeddings](/docs/aigw/integrations/llms/openai/embeddings.md)
- [Create embedding](/docs/aigw/api-reference/embeddings/create-embedding.md)
- [Guardrails for Embedding Requests](/docs/aigw/product/guardrails/embedding-guardrails.md)
- [Langchain (JS/TS)](/docs/aigw/integrations/libraries/langchain-js.md)