> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Fireworks

Portkey provides a robust and secure gateway to facilitate the integration of various models into your apps, including [chat](/integrations/llms/fireworks#id-3.-invoke-chat-completions-with-fireworks), [vision](/integrations/llms/fireworks#using-vision-models), [image generation](/integrations/llms/fireworks#using-image-generation-models), and [embedding](/integrations/llms/fireworks#using-embeddings-models) models hosted on the [Fireworks platform](https://fireworks.ai/).

With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/product/ai-gateway/virtual-keys) system.

<Note>
  Provider Slug. `fireworks-ai`
</Note>

## Portkey SDK Integration with Fireworks Models

Portkey provides a consistent API to interact with models from various providers. To integrate Fireworks with Portkey:

### 1. Install the Portkey SDK

<Tabs>
  <Tab title="NodeJS">
    ```sh theme={"system"}
    npm install --save portkey-ai
    ```
  </Tab>

  <Tab title="Python">
    ```sh theme={"system"}
    pip install portkey-ai
    ```
  </Tab>
</Tabs>

### 2. Initialize Portkey with the Virtual Key

To use Fireworks with Portkey, [get your API key from here](https://fireworks.ai/api-keys), then add it to Portkey to create the virtual key.

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    import Portkey from 'portkey-ai'

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY", // Defaults to process.env["PORTKEY_API_KEY"]
        virtualKey: "FIREWORKS_VIRTUAL_KEY" // Your Virtual Key
    })
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",  # Defaults to os.env("PORTKEY_API_KEY")
        virtual_key="FIREWORKS_VIRTUAL_KEY"   # Your Virtual Key
    )
    ```
  </Tab>
</Tabs>

### **3. Invoke Chat Completions with** Fireworks

You can use the Portkey instance now to send requests to Fireworks API.

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'accounts/fireworks/models/llama-v3-70b-instruct',
    });

    console.log(chatCompletion.choices);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    completion = portkey.chat.completions.create(
        messages= [{ "role": 'user', "content": 'Say this is a test' }],
        model= 'accounts/fireworks/models/llama-v3-70b-instruct'
    )

    print(completion)
    ```
  </Tab>
</Tabs>

Now, let's explore how you can use Portkey to call other models (vision, embedding, image) on the Fireworks API:

### Using Embeddings Models

Call any [embedding model hosted on Fireworks](https://readme.fireworks.ai/docs/querying-embeddings-models#list-of-available-models) with the familiar OpenAI embeddings signature:

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    const embeddings = await portkey.embeddings.create({
        input: "create vector representation on this sentence",
        model: "thenlper/gte-large",
    });

    console.log(embeddings);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    embeddings = portkey.embeddings.create(
        input='create vector representation on this sentence',
        model='thenlper/gte-large'
    )

    print(embeddings)
    ```
  </Tab>
</Tabs>

### Using Vision Models

Portkey natively supports [vision models hosted on Fireworks](https://readme.fireworks.ai/docs/querying-vision-language-models):

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    const completion = await portkey.chat.completions.create(
        messages: [
            { "role": "user", "content": [
                { "type": "text","text": "Can you describe this image?" },
                    { "type": "image_url", "image_url":
                        { "url": "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=1770&q=80" }
                    }
                ]
            }
        ],
        model: 'accounts/fireworks/models/firellava-13b'
    )

    console.log(completion);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    completion = portkey.chat.completions.create(
        messages= [
            { "role": "user", "content": [
                { "type": "text","text": "Can you describe this image?" },
                    { "type": "image_url", "image_url":
                        { "url": "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=1770&q=80" }
                    }
                ]
            }
        ],
        model= 'accounts/fireworks/models/firellava-13b'
    )

    print(completion)
    ```
  </Tab>
</Tabs>

### Using Image Generation Models

Portkey also supports calling [image generation models hosted on Fireworks](https://readme.fireworks.ai/reference/image%5Fgenerationaccountsfireworksmodelsstable-diffusion-xl-1024-v1-0) in the familiar OpenAI signature:

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    import Portkey from 'portkey-ai';
    import fs from 'fs';

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY",
        virtualKey: "FIREWORKS_VIRTUAL_KEY"
    });

    async function main(){
        const image = await portkey.images.generate({
            model: "accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
            prompt: "An orange elephant in a purple pond"
        });

        const imageData = image.data[0].b64_json as string;

        fs.writeFileSync("fireworks-image-gen.png", Buffer.from(imageData, 'base64'));
    }

    main()
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    from portkey_ai import Portkey
    import base64
    from io import BytesIO
    from PIL import Image

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",
        virtual_key="FIREWORKS_VIRTUAL_KEY"
    )

    image = portkey.images.generate(
      model="accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
      prompt="An orange elephant in a purple pond"
    )

    Image.open(BytesIO(base64.b64decode(image.data[0].b64_json))).save("fireworks-image-gen.png")
    ```
  </Tab>
</Tabs>

***

## Fireworks Grammar Mode

Fireworks lets you define [formal grammars](https://en.wikipedia.org/wiki/Formal%5Fgrammar) to constrain model outputs. You can use it to force the model to generate valid JSON, speak only in emojis, or anything else. ([Originally created by GGML](https://github.com/ggerganov/llama.cpp/tree/master/grammars))

Grammar mode is set with the `response_format` param. Just pass your grammar definition with `{"type": "grammar", "grammar": grammar_definition}`

Let's say you want to classify patient requests into 3 pre-defined classes:

<Tabs>
  <Tab title="Python">
    ```python theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",  # Defaults to os.env("PORTKEY_API_KEY")
        virtual_key="FIREWORKS_VIRTUAL_KEY"   # Your Virtual Key
    )

    patient_classification = """
    root      ::= diagnosis
    diagnosis ::= "flu" | "dengue" | "malaria"
    """

    completion = portkey.chat.completions.create(
        messages= [{ "role": 'user', "content": 'Say this is a test' }],
        response_format={"type": "grammar", "grammar": patient_classification},
        model= 'accounts/fireworks/models/llama-v3-70b-instruct'
    )

    print(completion)
    ```
  </Tab>

  <Tab title="NodeJS">
    ```js theme={"system"}
    import Portkey from 'portkey-ai'

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY", // Defaults to process.env["PORTKEY_API_KEY"]
        virtualKey: "FIREWORKS_VIRTUAL_KEY" // Your Virtual Key
    })

    const patient_classification = `
    root ::= diagnosis
    diagnosis ::= "flu" | "dengue" | "malaria"
    `;

    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        response_format: {"type": "grammar", "grammar": patient_classification},
        model: 'accounts/fireworks/models/llama-v3-70b-instruct',
    });

    console.log(chatCompletion.choices);
    ```
  </Tab>
</Tabs>

<Note>
  NOTE: Fireworks Grammer Mode is not supported on Portkey prompts playground
</Note>

[Explore the Fireworks guide for more examples and a deeper dive on Grammer node](https://readme.fireworks.ai/docs/structured-output-grammar-based).

## Fireworks JSON Mode

You can force the model to return (1) **An arbitrary JSON**, or (2) **JSON with given schema** with Fireworks' JSON mode.

<Tabs>
  <Tab title="Python">
    ```python theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",  # Defaults to os.env("PORTKEY_API_KEY")
        virtual_key="FIREWORKS_VIRTUAL_KEY"   # Your Virtual Key
    )

    class Recipe(BaseModel):
        title: str
        description: str
        steps: List[str]

    json_response = portkey.chat.completions.create(
        messages = [{ "role": 'user', "content": 'Give me a recipe for making Ramen, in JSON format' }],
        model = 'accounts/fireworks/models/llama-v3-70b-instruct',
        response_format = {
            "type":"json_object",
            "schema": Recipe.schema_json()
        }
    )

    print(json_response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="NodeJS">
    ```js theme={"system"}
    import Portkey from 'portkey-ai'

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY", // Defaults to process.env["PORTKEY_API_KEY"]
        virtualKey: "FIREWORKS_VIRTUAL_KEY" // Your Virtual Key
    })

    asyn function main(){
      const json_response = await portkey.chat.completions.create({
        messages: [{role: "user",content: `Give me a recipe for making Ramen, in JSON format`}],
        model: "accounts/fireworks/models/llama-v3-70b-instruct",
        response_format: {
          type: "json_object",
          schema: {
            type: "object",
            properties: {
              title: { type: "string" },
              description: { type: "string" },
              steps: { type: "array" }
            }
          }
        }
      });
    }

    console.log(json_response.choices[0].message.content);

    main()
    ```
  </Tab>
</Tabs>

[Explore Fireworks docs for JSON mode for more examples](https://readme.fireworks.ai/docs/structured-response-formatting).

## Fireworks Function Calling

Portkey also supports function calling mode on Fireworks. [Explore this cookbook for a deep dive and examples](/guides/getting-started/function-calling).

## Managing Fireworks Prompts

You can manage all Fireworks prompts in the [Prompt Library](/product/prompt-library). All the current 49+ language models available on Fireworks are supported and you can easily start testing different prompts.

Once you're ready with your prompt, you can use the `portkey.prompts.completions.create` interface to use the prompt in your application.

## Next Steps

The complete list of features supported in the SDK are available on the link below.

<Card title="SDK" href="/api-reference/portkey-sdk-client" />

You'll find more information in the relevant sections:

1. [Add metadata to your requests](/product/observability/metadata)
2. [Add gateway configs to your ](/product/ai-gateway/configs)[requests](/product/ai-gateway/configs)
3. [Tracing requests](/product/observability/traces)
4. [Setup a fallback from OpenAI to Firework APIs](/product/ai-gateway/fallbacks)