> ## Documentation Index
> Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Lepton AI

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Lepton AI APIs](https://www.lepton.ai/docs/guides).

With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/product/ai-gateway/virtual-keys) system.

<Note>
  Provider Slug. `lepton`
</Note>

## Portkey SDK Integration with Lepton AI Models

Portkey provides a consistent API to interact with models from various providers. To integrate Lepton AI with Portkey:

### 1. Install the Portkey SDK

Add the Portkey SDK to your application to interact with Lepton AI's API through Portkey's gateway.

<Tabs>
  <Tab title="NodeJS">
    ```sh theme={"system"}
    npm install --save portkey-ai
    ```
  </Tab>

  <Tab title="Python">
    ```sh theme={"system"}
    pip install portkey-ai
    ```
  </Tab>
</Tabs>

### 2. Initialize Portkey with the Virtual Key

To use Lepton AI with Portkey, [get your API key from Lepton AI](https://console.lepton.ai/), then add it to Portkey to create the virtual key.

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    import Portkey from 'portkey-ai'

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
        virtualKey: "VIRTUAL_KEY" // Your Lepton AI Virtual Key
    })
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
        virtual_key="VIRTUAL_KEY"   # Replace with your virtual key for Lepton
    )
    ```
  </Tab>
</Tabs>

### 3. Invoke Chat Completions with Lepton AI

Use the Portkey instance to send requests to Lepton AI. You can also override the virtual key directly in the API call if needed.

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'llama-3-8b-sft-v1',
    });

    console.log(chatCompletion.choices);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    completion = portkey.chat.completions.create(
        messages= [{ "role": 'user', "content": 'Say this is a test' }],
        model= 'llama-3-8b-sft-v1'
    )

    print(completion)
    ```
  </Tab>

  <Tab title="cURL">
    ```sh theme={"system"}
    curl --location 'https://api.portkey.ai/v1/chat/completions' \
    -H 'Content-Type: application/json' \
    -H 'x-portkey-api-key: PORTKEY_API_KEY' \
    -H 'x-portkey-virtual-key: VIRTUAL_KEY' \
    --data '{
        "model": "llama-3-8b-sft-v1",
        "messages": [
            {
                "role": "user",
                "content": "Say this is a test"
            }
        ]
    }'
    ```
  </Tab>
</Tabs>

## Speech-to-Text (Transcription)

Lepton AI provides speech-to-text capabilities through Portkey's unified API:

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    import fs from 'fs';

    const transcription = await portkey.audio.transcriptions.create({
        file: fs.createReadStream('audio.mp3'),
        model: 'whisper-large-v3',
    });

    console.log(transcription.text);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    with open("audio.mp3", "rb") as audio_file:
        transcription = portkey.audio.transcriptions.create(
            file=audio_file,
            model="whisper-large-v3"
        )

    print(transcription.text)
    ```
  </Tab>
</Tabs>

## Advanced Features

### Streaming Responses

Lepton AI supports streaming responses to provide real-time generation:

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    const stream = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Write a story about a robot' }],
        model: 'llama-3-8b-sft-v1',
        stream: true,
    });

    for await (const chunk of stream) {
        process.stdout.write(chunk.choices[0]?.delta?.content || '');
    }
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python theme={"system"}
    stream = portkey.chat.completions.create(
        messages=[{"role": "user", "content": "Write a story about a robot"}],
        model="llama-3-8b-sft-v1",
        stream=True
    )

    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")
    ```
  </Tab>
</Tabs>

## Managing Lepton AI Prompts

You can manage all prompts to Lepton AI in the [Prompt Library](/product/prompt-library). All the current models of Lepton AI are supported and you can easily start testing different prompts.

Once you're ready with your prompt, you can use the `portkey.prompts.completions.create` interface to use the prompt in your application.

## Next Steps

The complete list of features supported in the SDK are available on the link below.

<Card title="SDK" href="/api-reference/portkey-sdk-client" />

You'll find more information in the relevant sections:

1. [Add metadata to your requests](/product/observability/metadata)
2. [Add gateway configs to your Lepton AI requests](/product/ai-gateway/configs)
3. [Tracing Lepton AI requests](/product/observability/traces)
4. [Setup a fallback from OpenAI to Lepton AI APIs](/product/ai-gateway/fallbacks)