# Portkey Docs

## Docs

- [Get cache hit latency data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-cache-hit-latency-data.md)
- [Get cache hit rate data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-cache-hit-rate-data.md)
- [Get cost data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-cost-data.md)
- [Get error rate data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-error-rate-data.md)
- [Get errors data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-errors-data.md)
- [Get feedback data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-feedback-data.md)
- [Get feedback per ai models data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-feedback-per-ai-models-data.md)
- [Get feedback score distribution data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-feedback-score-distribution-data.md)
- [Get latency data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-latency-data.md)
- [Get requests data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-requests-data.md)
- [Get requests per user data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-requests-per-user-data.md)
- [Get rescued requests data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-rescued-requests-data.md)
- [Get status code data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-status-code-data.md)
- [Get tokens data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-tokens-data.md)
- [Get unique status code data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-unique-status-code-data.md)
- [Get users data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-users-data.md)
- [Get weighted feedback data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-weighted-feedback-data.md)
- [Get Metadata Grouped Data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/groups-paginated-data/get-metadata-grouped-data.md)
- [Get Model Grouped Data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/groups-paginated-data/get-model-grouped-data.md)
- [Get User Grouped Data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/groups-paginated-data/get-user-grouped-data.md)
- [Get User Grouped Data](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/analytics/summary/get-all-cache-data.md)
- [Create API Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/api-keys/create-api-key.md)
- [Delete an API Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/api-keys/delete-an-api-key.md)
- [List API Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/api-keys/list-api-keys.md)
- [Retrieve and API Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/api-keys/retrieve-an-api-key.md)
- [Update API Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/api-keys/update-api-key.md)
- [Create Config](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/configs/create-config.md)
- [List Configs](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/configs/list-configs.md)
- [Retrieve Config](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/configs/retrieve-config.md)
- [Update Config](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/configs/update-config.md)
- [Delete a user invite](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/user-invites/delete-a-user-invite.md)
- [Invite a User](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/user-invites/invite-a-user.md): Send an invite to user for your organization
- [Resend a user invite](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/user-invites/resend-a-user-invite.md): Resend an invite to user for your organization
- [Retrieve all user invite](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/user-invites/retrieve-all-user-invites.md)
- [Retrieve an user invite](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/user-invites/retrieve-an-invite.md)
- [Remove a user](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/users/remove-a-user.md)
- [Retrieve a user](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/users/retrieve-a-user.md)
- [Retrieve all users](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/users/retrieve-all-users.md)
- [Update a user](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/users/update-a-user.md)
- [Create Virtual Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/create-virtual-key.md)
- [Delete Virtual Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/delete-virtual-key.md)
- [List Virtual Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/list-virtual-keys.md)
- [Retrieve Virtual Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/retrieve-virtual-key.md)
- [Update Virtual Key](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/virtual-keys/update-virtual-key.md)
- [Add a Workspace Member](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspace-members/add-a-workspace-member.md)
- [Remove Workspace Member](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspace-members/remove-workspace-member.md)
- [Retrieve a Workspace Member](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspace-members/retrieve-a-workspace-member.md)
- [Retrieve all Workspace Member](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspace-members/retrieve-all-workspace-members.md)
- [Update Workspace Member](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspace-members/update-workspace-member.md)
- [Create Workspace](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspaces/create-workspace.md)
- [Delete a Workspace](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspaces/delete-a-workspace.md)
- [Retrieve a Workspace](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspaces/retrieve-a-workspace.md)
- [Retrieve all Workspaces](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspaces/retrieve-all-workspaces.md)
- [Update Workspace](https://docs.portkey.ai/docs/api-reference/admin-api/control-plane/workspaces/update-workspace.md)
- [Feedback](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/feedback.md): Feedback in Portkey provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app.
- [Create Feedback](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/feedback/create-feedback.md): This endpoint allows users to submit feedback for a particular interaction or response.
- [Update Feedback](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/feedback/update-feedback.md): This endpoint allows users to update existing feedback.
- [Guardrails](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/guardrails.md)
- [Insert a Log](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/insert-a-log.md): Submit one or more log entries
- [Cancel a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/cancel-a-log-export.md)
- [Create a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/create-a-log-export.md)
- [Download a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/download-a-log-export.md)
- [List a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/list-log-exports.md)
- [Retrieve a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/retrieve-a-log-export.md)
- [Start a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/start-a-log-export.md)
- [Update a Log Export](https://docs.portkey.ai/docs/api-reference/admin-api/data-plane/logs/log-exports-beta/update-a-log-export.md)
- [Introduction](https://docs.portkey.ai/docs/api-reference/admin-api/introduction.md): Manage your Portkey organization and workspaces programmatically
- [OpenAPI Specification](https://docs.portkey.ai/docs/api-reference/admin-api/open-api-specification.md)
- [null](https://docs.portkey.ai/docs/api-reference/inference-api/anthropic-transform.md)
- [Create Assistant](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/assistants/create-assistant.md)
- [Delete Assistant](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/assistants/delete-assistant.md)
- [List Assistant](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/assistants/list-assistants.md)
- [Modify Assistant](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/assistants/modify-assistant.md)
- [Retrieve Assistant](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/assistants/retrieve-assistant.md)
- [Create Message](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/messages/create-message.md)
- [Delete Message](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/messages/delete-message.md)
- [List Message](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/messages/list-messages.md)
- [Modify Message](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/messages/modify-message.md)
- [Retrieve Message](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/messages/retrieve-message.md)
- [List Run Steps](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/run-steps/list-run-steps.md)
- [Retrieve Run Steps](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/run-steps/retrieve-run-steps.md)
- [Cancel Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/cancel-run.md)
- [Create Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/create-run.md)
- [Create thread and Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/create-thread-and-run.md)
- [list Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/list-runs.md)
- [Modify Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/modify-run.md)
- [Retrieve Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/retrieve-run.md)
- [Submit Tool Outputs to Run](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/runs/submit-tool-outputs-to-run.md)
- [Create Thread](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/threads/create-thread.md)
- [Delete Thread](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/threads/delete-thread.md)
- [Modify Thread](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/threads/modify-thread.md)
- [Retrieve Thread](https://docs.portkey.ai/docs/api-reference/inference-api/assistants-api/threads/retrieve-thread.md)
- [Create Speech](https://docs.portkey.ai/docs/api-reference/inference-api/audio/create-speech.md)
- [Create Transcription](https://docs.portkey.ai/docs/api-reference/inference-api/audio/create-transcription.md)
- [Create Translation](https://docs.portkey.ai/docs/api-reference/inference-api/audio/create-translation.md)
- [Authentication](https://docs.portkey.ai/docs/api-reference/inference-api/authentication.md)
- [Cancel Batch](https://docs.portkey.ai/docs/api-reference/inference-api/batch/cancel-batch.md)
- [Create Batch](https://docs.portkey.ai/docs/api-reference/inference-api/batch/create-batch.md)
- [List Batch](https://docs.portkey.ai/docs/api-reference/inference-api/batch/list-batch.md)
- [Retrieve Batch](https://docs.portkey.ai/docs/api-reference/inference-api/batch/retrieve-batch.md)
- [Chat](https://docs.portkey.ai/docs/api-reference/inference-api/chat.md)
- [Completions](https://docs.portkey.ai/docs/api-reference/inference-api/completions.md)
- [Gateway Config Object](https://docs.portkey.ai/docs/api-reference/inference-api/config-object.md)
- [Embeddings](https://docs.portkey.ai/docs/api-reference/inference-api/embeddings.md)
- [Errors](https://docs.portkey.ai/docs/api-reference/inference-api/error-codes.md)
- [Delete File](https://docs.portkey.ai/docs/api-reference/inference-api/files/delete-file.md)
- [List Files](https://docs.portkey.ai/docs/api-reference/inference-api/files/list-files.md)
- [Retrieve File](https://docs.portkey.ai/docs/api-reference/inference-api/files/retrieve-file.md)
- [Retrieve File Content](https://docs.portkey.ai/docs/api-reference/inference-api/files/retrieve-file-content.md)
- [Upload File](https://docs.portkey.ai/docs/api-reference/inference-api/files/upload-file.md)
- [Cancel Fine-tuning](https://docs.portkey.ai/docs/api-reference/inference-api/fine-tuning/cancel-fine-tuning.md)
- [Create Fine-tuning Job](https://docs.portkey.ai/docs/api-reference/inference-api/fine-tuning/create-fine-tuning-job.md): Finetune a provider model
- [List Fine-tuning Checkpoints](https://docs.portkey.ai/docs/api-reference/inference-api/fine-tuning/list-fine-tuning-checkpoints.md)
- [List Fine-tuning Events](https://docs.portkey.ai/docs/api-reference/inference-api/fine-tuning/list-fine-tuning-events.md)
- [List Fine-tuning Jobs](https://docs.portkey.ai/docs/api-reference/inference-api/fine-tuning/list-fine-tuning-jobs.md)
- [Retrieve Fine-tuning Job](https://docs.portkey.ai/docs/api-reference/inference-api/fine-tuning/retrieve-fine-tuning-job.md)
- [Gateway to Other APIs](https://docs.portkey.ai/docs/api-reference/inference-api/gateway-for-other-apis.md): Access any custom provider endpoint through Portkey API
- [Headers](https://docs.portkey.ai/docs/api-reference/inference-api/headers.md): Header requirements and options for the Portkey API
- [Create Image](https://docs.portkey.ai/docs/api-reference/inference-api/images/create-image.md)
- [Create Image Edit](https://docs.portkey.ai/docs/api-reference/inference-api/images/create-image-edit.md)
- [Create Image Variation](https://docs.portkey.ai/docs/api-reference/inference-api/images/create-image-variation.md)
- [Introduction](https://docs.portkey.ai/docs/api-reference/inference-api/introduction.md): This documentation provides detailed information about the various ways you can access and interact with Portkey - **a robust AI gateway** designed to simplify and enhance your experience with Large Language Models (LLMs) like OpenAI's GPT models. 
- [Moderations](https://docs.portkey.ai/docs/api-reference/inference-api/moderations.md)
- [OpenAPI Specification](https://docs.portkey.ai/docs/api-reference/inference-api/open-api-specification.md)
- [Python & Node](https://docs.portkey.ai/docs/api-reference/inference-api/portkey-sdk-client.md): The Portkey SDK client enables various features of Portkey in an easy to use `config-as-code` paradigm.
- [Prompt Completions](https://docs.portkey.ai/docs/api-reference/inference-api/prompts/prompt-completion.md): Execute your saved prompt templates on Portkey

- [Prompt Render](https://docs.portkey.ai/docs/api-reference/inference-api/prompts/render.md): Renders a prompt template with its variable values filled in

- [Response Schema](https://docs.portkey.ai/docs/api-reference/inference-api/response-schema.md)
- [C# (.NET)](https://docs.portkey.ai/docs/api-reference/inference-api/sdks/c-sharp.md): Integrate Portkey in your `.NET` app easily using the OpenAI library and get advanced monitoring, routing, and enterprise features.
- [Supported Libraries](https://docs.portkey.ai/docs/api-reference/inference-api/sdks/supported-sdks.md): Use Portkey APIs in your preferred programming language
- [Supported Providers](https://docs.portkey.ai/docs/api-reference/inference-api/supported-providers.md)
- [Portkey SDK](https://docs.portkey.ai/docs/api-reference/portkey-sdk-client.md): The Portkey SDK client enables various features of Portkey in an easy to use `config-as-code` paradigm.
- [December](https://docs.portkey.ai/docs/changelog/2024/dec.md)
- [November](https://docs.portkey.ai/docs/changelog/2024/nov.md)
- [October](https://docs.portkey.ai/docs/changelog/2024/oct.md)
- [February](https://docs.portkey.ai/docs/changelog/2025/feb.md)
- [January](https://docs.portkey.ai/docs/changelog/2025/jan.md)
- [March](https://docs.portkey.ai/docs/changelog/2025/mar.md)
- [null](https://docs.portkey.ai/docs/changelog/2025/todo.md)
- [Enterprise Gateway](https://docs.portkey.ai/docs/changelog/enterprise.md)
- [Helm Chart](https://docs.portkey.ai/docs/changelog/helm-chart.md)
- [AI Engineering Hours](https://docs.portkey.ai/docs/changelog/office-hour.md): Discussion notes from the weekly AI engineering meetup
- [OSS Gateway](https://docs.portkey.ai/docs/changelog/open-source.md)
- [Latest Updates](https://docs.portkey.ai/docs/changelog/product.md)
- [Overview](https://docs.portkey.ai/docs/guides/getting-started.md)
- [101 on Portkey's Gateway Configs](https://docs.portkey.ai/docs/guides/getting-started/101-on-portkey-s-gateway-configs.md): You are likely familiar with how to make an API call to GPT4 for chat completions. 
- [A/B Test Prompts and Models](https://docs.portkey.ai/docs/guides/getting-started/a-b-test-prompts-and-models.md): A/B testing with large language models in production is crucial for driving optimal performance and user satisfaction.
- [Function Calling](https://docs.portkey.ai/docs/guides/getting-started/function-calling.md): Get the LLM to interact with external APIs!
- [Getting started with AI Gateway](https://docs.portkey.ai/docs/guides/getting-started/getting-started-with-ai-gateway.md)
- [Image Generation](https://docs.portkey.ai/docs/guides/getting-started/image-generation.md)
- [Llama 3 on Groq](https://docs.portkey.ai/docs/guides/getting-started/llama-3-on-groq.md)
- [Return Repeat Requests from Cache](https://docs.portkey.ai/docs/guides/getting-started/return-repeat-requests-from-cache.md): If you have multiple users of your GenAI app triggering the same or similar queries to your models, fetching LLM response from the models can be slow and expensive.
- [Tackling Rate Limiting](https://docs.portkey.ai/docs/guides/getting-started/tackling-rate-limiting.md): LLMs are **costly** to run. As their usage increaases, the providers have to balance serving user requests v/s straining their GPU resources too thin. They generally deal with this by putting _rate limits_ on how many requests a user can send in a minute or in a day.
- [Trigger Automatic Retries on LLM Failures](https://docs.portkey.ai/docs/guides/getting-started/trigger-automatic-retries-on-llm-failures.md)
- [Overview](https://docs.portkey.ai/docs/guides/integrations.md)
- [Anyscale](https://docs.portkey.ai/docs/guides/integrations/anyscale.md): Portkey helps bring Anyscale APIs to production with its abstractions for observability, fallbacks, caching, and more. Use the Anyscale API **through** Portkey for.
- [Deepinfra](https://docs.portkey.ai/docs/guides/integrations/deepinfra.md)
- [Groq](https://docs.portkey.ai/docs/guides/integrations/groq.md)
- [Introduction to GPT-4o](https://docs.portkey.ai/docs/guides/integrations/introduction-to-gpt-4o.md)
- [Langchain](https://docs.portkey.ai/docs/guides/integrations/langchain.md)
- [Llama 3 on Portkey + Together AI](https://docs.portkey.ai/docs/guides/integrations/llama-3-on-portkey-+-together-ai.md): Try out the new Llama 3 model directly using the OpenAI SDK
- [Mistral](https://docs.portkey.ai/docs/guides/integrations/mistral.md): Portkey helps bring Mistral's APIs to production with its observability suite & AI Gateway.
- [Mixtral 8x22b](https://docs.portkey.ai/docs/guides/integrations/mixtral-8x22b.md)
- [Segmind](https://docs.portkey.ai/docs/guides/integrations/segmind.md)
- [Vercel AI](https://docs.portkey.ai/docs/guides/integrations/vercel-ai.md): Portkey is a control panel for your Vercel AI app. It makes your LLM integrations prod-ready, reliable, fast, and cost-efficient.
- [null](https://docs.portkey.ai/docs/guides/prompts.md)
- [Build a chatbot using Portkey's Prompt Templates](https://docs.portkey.ai/docs/guides/prompts/build-a-chatbot-using-portkeys-prompt-templates.md): Portkey's prompt templates offer a powerful solution for testing and building chatbots.
- [Building an LLM-as-a-Judge System for AI (Customer Support) Agent](https://docs.portkey.ai/docs/guides/prompts/llm-as-a-judge.md)
- [Ultimate AI SDR](https://docs.portkey.ai/docs/guides/prompts/ultimate-ai-sdr.md): Building a sophisticated AI SDR agent leveraging internet search and evals to draft personalized outreach emails in 15 seconds
- [Portkey at TEDAI Hackathon 2024](https://docs.portkey.ai/docs/guides/ted-ai-hack-24.md)
- [Overview](https://docs.portkey.ai/docs/guides/use-cases.md)
- [Build an article suggestion app with Supabase pgvector, and Portkey](https://docs.portkey.ai/docs/guides/use-cases/build-an-article-suggestion-app-with-supabase-pgvector-and-portkey.md):  Consider that you have list of support articles that you want to suggest it to users when users search for it. You want to suggest as best fit as possible. With the availability of tools like Large Language Model (LLMs) and Vector Databases, the approach towards suggestions & recommendation systems has significantly evolved.
- [Comparing Top10 LMSYS Models with Portkey](https://docs.portkey.ai/docs/guides/use-cases/comparing-top10-lmsys-models-with-portkey.md)
- [Comparing DeepSeek Models Against OpenAI, Anthropic & More Using Portkey](https://docs.portkey.ai/docs/guides/use-cases/deepseek-r1.md)
- [Detecting Emotions with GPT-4o](https://docs.portkey.ai/docs/guides/use-cases/emotions-with-gpt-4o.md)
- [Enforcing JSON Schema with Anyscale & Together](https://docs.portkey.ai/docs/guides/use-cases/enforcing-json-schema-with-anyscale-and-together.md): Get the LLM to adhere to your JSON schema using Anyscale & Together AI's newly introduced JSON modes
- [Fallback from SDXL to Dall-e-3](https://docs.portkey.ai/docs/guides/use-cases/fallback-from-sdxl-to-dall-e-3.md): Generative AI models have revolutionized text generation and opened up new possibilities for developers.
- [Few-Shot Prompting](https://docs.portkey.ai/docs/guides/use-cases/few-shot-prompting.md): LLMs are highly capable of following a given structure. By providing a few examples of how the assistant should respond to a given prompt, the LLM can generate responses that closely follow the format of these examples.
- [How to use OpenAI SDK with Portkey Prompt Templates](https://docs.portkey.ai/docs/guides/use-cases/how-to-use-openai-sdk-with-portkey-prompt-templates.md): Portkeys Prompt Playground allows you to test and tinker with various hyperparameters without any external dependencies and deploy them to production seamlessly. Moreover, all team members can use the same prompt template, ensuring that everyone works from the same source of truth.
- [Run Portkey on Prompts from Langchain Hub](https://docs.portkey.ai/docs/guides/use-cases/run-portkey-on-prompts-from-langchain-hub.md): Writing the right prompt is often hard to get a quality LLM response. You want the prompt to be specialized and exhaustive enough for your problem. There is a high chance someone else might’ve stumbled across a similar situation and written the prompt you’ve been figuring out all this while. 
- [Setting up resilient Load balancers with failure-mitigating Fallbacks](https://docs.portkey.ai/docs/guides/use-cases/setting-up-resilient-load-balancers-with-failure-mitigating-fallbacks.md): Companies often face challenges of scaling their services efficiently as the traffic to their applications grow - when you’re consuming APIs, the first point of failure is that if you hit the API too much, you can get rate limited. Loadbalancing is a proven way to scale usage horizontally without overburdening any one provider and thus staying within rate limits.
- [Setup OpenAI -> Azure OpenAI Fallback](https://docs.portkey.ai/docs/guides/use-cases/setup-openai-greater-than-azure-openai-fallback.md): Portkey Fallbacks can automatically switch your app's requests from one LLM provider to another, ensuring reliability by allowing you to fallback among multiple LLMs.
- [Smart Fallback with Model-Optimized Prompts](https://docs.portkey.ai/docs/guides/use-cases/smart-fallback-with-model-optimized-prompts.md): Portkey can help you easily create fallbacks from one LLM to another, making your application more reliable. While Fallback ensures reliability, it also means that you'll be running a prompt optimized for one LLM on another, which can often lead to significant differences in the final output.
- [Tracking LLM Costs Per User with Portkey](https://docs.portkey.ai/docs/guides/use-cases/track-costs-using-metadata.md): Monitor and analyze user-level LLM costs across 1600+ models using Portkey's metadata and analytics API.
- [4. Advanced Strategies for Performance Improvement](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/advanced-strategies.md)
- [5. Architectural Considerations](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/architectural-considerations.md)
- [10. Conclusion and Key Takeaways](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/conclusion-and-key-takeaways.md): Summarizing the key strategies for LLM cost optimization and performance improvement
- [7. Cost Effective Development Practices](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/cost-effective-development.md)
- [Executive Summary](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/executive-summary.md): Overview of LLM cost optimization and performance improvement strategies
- [3. FrugalGPT Techniques for Cost Optimization](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/frugalgpt-techniques.md)
- [9. Future Trends in LLM Cost Optimization](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/future-trends.md)
- [1. Introduction](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/introduction.md): An overview of the challenges and opportunities in LLM cost optimization
- [2. Understanding LLM Cost Drivers](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/llm-cost-drivers.md): An overview of the factors that influence costs in Large Language Model applications
- [6. Operational Best Pracitces](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/operational-best-practices.md)
- [8. User Education and Change Management](https://docs.portkey.ai/docs/guides/whitepapers/optimizing-llm-costs/user-education.md)
- [Integrations](https://docs.portkey.ai/docs/integrations.md)
- [Overview](https://docs.portkey.ai/docs/integrations/agents.md): Portkey helps bring your agents to production
- [Autogen](https://docs.portkey.ai/docs/integrations/agents/autogen.md): Use Portkey with Autogen to take your AI Agents to production
- [Bring Your own Agents](https://docs.portkey.ai/docs/integrations/agents/bring-your-own-agents.md): You can also use Portkey if you are doing custom agent orchestration!
- [Control Flow](https://docs.portkey.ai/docs/integrations/agents/control-flow.md): Use Portkey with Control Flow to take your AI Agents to production
- [CrewAI](https://docs.portkey.ai/docs/integrations/agents/crewai.md): Use Portkey with CrewAI to take your AI Agents to production
- [Langchain Agents](https://docs.portkey.ai/docs/integrations/agents/langchain-agents.md)
- [LangGraph Agents](https://docs.portkey.ai/docs/integrations/agents/langgraph.md): Use Portkey with LangGraph to take your AI Agents to production
- [Llama Agents by Llamaindex](https://docs.portkey.ai/docs/integrations/agents/llama-agents.md): Use Portkey with Llama Agents to take your AI Agents to production
- [OpenAI Swarm](https://docs.portkey.ai/docs/integrations/agents/openai-swarm.md): The Portkey x Swarm integration brings advanced AI gateway capabilities, full-stack observability, and reliability features to build production-ready AI agents.
- [Phidata](https://docs.portkey.ai/docs/integrations/agents/phidata.md): Use Portkey with Phidata to take your AI Agents to production
- [Integrations](https://docs.portkey.ai/docs/integrations/ecosystem.md)
- [Overview](https://docs.portkey.ai/docs/integrations/libraries.md)
- [Anything LLM](https://docs.portkey.ai/docs/integrations/libraries/anythingllm.md): Add usage tracking, cost controls, and security guardrails to your Anything LLM deployment
- [Autogen](https://docs.portkey.ai/docs/integrations/libraries/autogen.md): AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.
- [DSPy](https://docs.portkey.ai/docs/integrations/libraries/dspy.md): Integrate DSPy with Portkey for production-ready LLM pipelines
- [Instructor](https://docs.portkey.ai/docs/integrations/libraries/instructor.md): With Portkey, you can confidently take your Instructor pipelines to production and get complete observability over all of your calls + make them reliable - all with a 2 LOC change!
- [Jan](https://docs.portkey.ai/docs/integrations/libraries/janhq.md): Add usage tracking, cost controls, and security guardrails to your Jan deployment
- [Langchain (JS/TS)](https://docs.portkey.ai/docs/integrations/libraries/langchain-js.md): Portkey adds core production capabilities to any Langchain app.
- [Langchain (Python)](https://docs.portkey.ai/docs/integrations/libraries/langchain-python.md): Portkey adds core production capabilities to any Langchain app.
- [LibreChat](https://docs.portkey.ai/docs/integrations/libraries/librechat.md): Cost tracking, observability, and more for LibreChat
- [LlamaIndex (Python)](https://docs.portkey.ai/docs/integrations/libraries/llama-index-python.md): The **Portkey x LlamaIndex** integration brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** to apps built on LlamaIndex.
- [Microsoft Semantic Kernel](https://docs.portkey.ai/docs/integrations/libraries/microsoft-semantic-kernel.md)
- [MindsDb](https://docs.portkey.ai/docs/integrations/libraries/mindsdb.md): Integrate MindsDB with Portkey to build enterprise-grade AI use-cases
- [MongoDB](https://docs.portkey.ai/docs/integrations/libraries/mongodb.md)
- [Portkey with Any OpenAI Compatible Project](https://docs.portkey.ai/docs/integrations/libraries/openai-compatible.md): Learn how to integrate Portkey's enterprise features with any OpenAI Compliant project for enhanced observability, reliability and governance.
- [Open WebUI](https://docs.portkey.ai/docs/integrations/libraries/openwebui.md): Cost tracking, observability, and more for Open WebUI
- [Promptfoo](https://docs.portkey.ai/docs/integrations/libraries/promptfoo.md): Portkey brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** + **versioning** to your **Promptfoo** projects. This document provides an overview of how to leverage the strengths of both the platforms to streamline your AI development workflow.
- [Supabase](https://docs.portkey.ai/docs/integrations/libraries/supabase.md)
- [ToolJet](https://docs.portkey.ai/docs/integrations/libraries/tooljet.md): ToolJet is a low-code platform that lets you build apps by connecting APIs and data sources, with Portkey integration adding AI features like chat interfaces and automation.
- [Vercel](https://docs.portkey.ai/docs/integrations/libraries/vercel.md): Integrate Portkey with Vercel AI SDK for production-ready and reliable AI apps
- [Zed](https://docs.portkey.ai/docs/integrations/libraries/zed.md): Learn how to integrate Portkey's enterprise features with Zed for enhanced observability, reliability and governance.
- [Overview](https://docs.portkey.ai/docs/integrations/llms.md): Portkey connects with all major LLM providers and orchestration frameworks.
- [AI21](https://docs.portkey.ai/docs/integrations/llms/ai21.md)
- [Anthropic](https://docs.portkey.ai/docs/integrations/llms/anthropic.md)
- [Prompt Caching](https://docs.portkey.ai/docs/integrations/llms/anthropic/prompt-caching.md)
- [Anyscale](https://docs.portkey.ai/docs/integrations/llms/anyscale-llama2-mistral-zephyr.md): Integrate Anyscale endpoints with Portkey seamlessly and make your OSS models production-ready
- [AWS SageMaker](https://docs.portkey.ai/docs/integrations/llms/aws-sagemaker.md): Route to your AWS Sagemaker models through Portkey
- [Azure OpenAI](https://docs.portkey.ai/docs/integrations/llms/azure-openai/azure-openai.md): Azure OpenAI is a great alternative to accessing the best models including GPT-4 and more in your private environments. Portkey provides complete support for Azure OpenAI.
- [Batches](https://docs.portkey.ai/docs/integrations/llms/azure-openai/batches.md): Perform batch inference with Azure OpenAI
- [Files](https://docs.portkey.ai/docs/integrations/llms/azure-openai/files.md): Upload files to Azure OpenAI
- [Fine-tune](https://docs.portkey.ai/docs/integrations/llms/azure-openai/fine-tuning.md): Fine-tune your models with Azure OpenAI
- [AWS Bedrock](https://docs.portkey.ai/docs/integrations/llms/bedrock/aws-bedrock.md)
- [Batches](https://docs.portkey.ai/docs/integrations/llms/bedrock/batches.md): Perform batch inference with Bedrock
- [Files](https://docs.portkey.ai/docs/integrations/llms/bedrock/files.md): Upload files to S3 for Bedrock batch inference
- [Fine-tune](https://docs.portkey.ai/docs/integrations/llms/bedrock/fine-tuning.md): Fine-tune your models with Bedrock
- [Bring Your Own LLM](https://docs.portkey.ai/docs/integrations/llms/byollm.md)
- [Cerebras](https://docs.portkey.ai/docs/integrations/llms/cerebras.md)
- [Cohere](https://docs.portkey.ai/docs/integrations/llms/cohere.md)
- [Dashscope](https://docs.portkey.ai/docs/integrations/llms/dashscope.md): Integrate Dashscope with Portkey for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning.
- [Deepbricks](https://docs.portkey.ai/docs/integrations/llms/deepbricks.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Deepbricks](https://deepbricks.ai/). 
- [Deepgram](https://docs.portkey.ai/docs/integrations/llms/deepgram.md): Portkey provides a robust and secure gateway to use and observe Deepgrm's Speech-to-Text API.
- [Deepinfra](https://docs.portkey.ai/docs/integrations/llms/deepinfra.md)
- [DeepSeek](https://docs.portkey.ai/docs/integrations/llms/deepseek.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including DeepSeek models. 
- [Fireworks](https://docs.portkey.ai/docs/integrations/llms/fireworks.md)
- [Files](https://docs.portkey.ai/docs/integrations/llms/fireworks/files.md): Upload files to Fireworks
- [Fine-tune](https://docs.portkey.ai/docs/integrations/llms/fireworks/fine-tuning.md): Fine-tune your models with Bedrock
- [Google Gemini](https://docs.portkey.ai/docs/integrations/llms/gemini.md)
- [Github](https://docs.portkey.ai/docs/integrations/llms/github.md)
- [Google Palm](https://docs.portkey.ai/docs/integrations/llms/google-palm.md)
- [Groq](https://docs.portkey.ai/docs/integrations/llms/groq.md)
- [Hugging Face](https://docs.portkey.ai/docs/integrations/llms/huggingface.md)
- [Inference.net](https://docs.portkey.ai/docs/integrations/llms/inference.net.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on [Inference.net](https://www.inference.net/). 
- [Jina AI](https://docs.portkey.ai/docs/integrations/llms/jina-ai.md)
- [Lambda Labs](https://docs.portkey.ai/docs/integrations/llms/lambda.md): Integrate Lambda with Portkey AI for seamless completions, prompt management, and advanced features like streaming and function calling.
- [Lemonfox-AI](https://docs.portkey.ai/docs/integrations/llms/lemon-fox.md): Integrate LemonFox with Portkey for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning.
- [Lingyi (01.ai)](https://docs.portkey.ai/docs/integrations/llms/lingyi-01.ai.md)
- [LocalAI](https://docs.portkey.ai/docs/integrations/llms/local-ai.md)
- [Mistral AI](https://docs.portkey.ai/docs/integrations/llms/mistral-ai.md)
- [Monster API](https://docs.portkey.ai/docs/integrations/llms/monster-api.md): MonsterAPIs provides access to generative AI model APIs at 80% lower costs. Connect to MonsterAPI LLM APIs seamlessly through Portkey's AI gateway.
- [Moonshot](https://docs.portkey.ai/docs/integrations/llms/moonshot.md)
- [Nebius](https://docs.portkey.ai/docs/integrations/llms/nebius.md)
- [Nomic](https://docs.portkey.ai/docs/integrations/llms/nomic.md)
- [Novita AI](https://docs.portkey.ai/docs/integrations/llms/novita-ai.md)
- [Ollama](https://docs.portkey.ai/docs/integrations/llms/ollama.md)
- [OpenAI](https://docs.portkey.ai/docs/integrations/llms/openai.md): Learn to integrate OpenAI with Portkey, enabling seamless completions, prompt management, and advanced functionalities like streaming, function calling and fine-tuning.
- [Batches](https://docs.portkey.ai/docs/integrations/llms/openai/batches.md): Perform batch inference with OpenAI
- [Files](https://docs.portkey.ai/docs/integrations/llms/openai/files.md): Upload files to OpenAI
- [Fine-tune](https://docs.portkey.ai/docs/integrations/llms/openai/fine-tuning.md): Fine-tune your models with OpenAI
- [Prompt Caching](https://docs.portkey.ai/docs/integrations/llms/openai/prompt-caching-openai.md)
- [Structured Outputs](https://docs.portkey.ai/docs/integrations/llms/openai/structured-outputs.md): Structured Outputs ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Portkey supports OpenAI's Structured Outputs feature out of the box with our SDKs & APIs. 
- [OpenAI](https://docs.portkey.ai/docs/integrations/llms/openai2.md): Integrate OpenAI with Portkey to get production metrics for your requests and make chat completions, audio, image generation, structured outputs, function calling, fine-tuning, batch, and more requests.
- [OpenAI](https://docs.portkey.ai/docs/integrations/llms/openai3.md): Complete guide to integrate OpenAI API with Portkey. Support for gpt-4o, o1, chat completions, vision, and audio APIs with built-in reliability and monitoring features.
- [OpenRouter](https://docs.portkey.ai/docs/integrations/llms/openrouter.md)
- [Perplexity AI](https://docs.portkey.ai/docs/integrations/llms/perplexity-ai.md)
- [Predibase](https://docs.portkey.ai/docs/integrations/llms/predibase.md)
- [Reka AI](https://docs.portkey.ai/docs/integrations/llms/reka-ai.md)
- [Replicate](https://docs.portkey.ai/docs/integrations/llms/replicate.md)
- [SambaNova](https://docs.portkey.ai/docs/integrations/llms/sambanova.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [SambaNova AI](https://sambanova.ai/).
- [Segmind](https://docs.portkey.ai/docs/integrations/llms/segmind.md)
- [SiliconFlow](https://docs.portkey.ai/docs/integrations/llms/siliconflow.md)
- [Stability AI](https://docs.portkey.ai/docs/integrations/llms/stability-ai.md)
- [Suggest a new integration!](https://docs.portkey.ai/docs/integrations/llms/suggest-a-new-integration.md)
- [Together AI](https://docs.portkey.ai/docs/integrations/llms/together-ai.md)
- [Triton](https://docs.portkey.ai/docs/integrations/llms/triton.md): Integrate Trtiton-hosted custom models with Portkey and take them to production
- [Upstage AI](https://docs.portkey.ai/docs/integrations/llms/upstage.md): Integrate Upstage with Portkey AI for seamless completions, prompt management, and advanced features like streaming and embedding.
- [Google Vertex AI](https://docs.portkey.ai/docs/integrations/llms/vertex-ai.md)
- [Batches](https://docs.portkey.ai/docs/integrations/llms/vertex-ai/batches.md): Perform batch inference with Vertex AI
- [Controlled Generations](https://docs.portkey.ai/docs/integrations/llms/vertex-ai/controlled-generations.md): Controlled Generations ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Portkey supports Vertex AI's Controlled Generations feature out of the box with our SDKs & APIs. 
- [Files](https://docs.portkey.ai/docs/integrations/llms/vertex-ai/files.md): Upload files to Google Cloud Storage for Vertex AI fine-tuning and batch inference
- [Fine-tune](https://docs.portkey.ai/docs/integrations/llms/vertex-ai/fine-tuning.md): Fine-tune your models with Vertex AI
- [vLLM](https://docs.portkey.ai/docs/integrations/llms/vllm.md): Integrate vLLM-hosted custom models with Portkey and take them to production
- [Voyage AI](https://docs.portkey.ai/docs/integrations/llms/voyage-ai.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including Voyage AI's embedding and Re-rank endpoints. 
- [Workers AI](https://docs.portkey.ai/docs/integrations/llms/workers-ai.md)
- [xAI](https://docs.portkey.ai/docs/integrations/llms/x-ai.md): Portkey supports xAI's chat completions, completions, and embeddings APIs.
- [ZhipuAI / ChatGLM / BigModel](https://docs.portkey.ai/docs/integrations/llms/zhipu.md)
- [Submit an Integration](https://docs.portkey.ai/docs/integrations/partner.md)
- [Milvus](https://docs.portkey.ai/docs/integrations/vector-databases/milvus.md)
- [Qdrant](https://docs.portkey.ai/docs/integrations/vector-databases/qdrant.md)
- [Portkey Features](https://docs.portkey.ai/docs/introduction/feature-overview.md): Explore the powerful features of Portkey
- [Make Your First Request](https://docs.portkey.ai/docs/introduction/make-your-first-request.md): Integrate Portkey and analyze your first LLM call in 2 minutes!
- [What is Portkey?](https://docs.portkey.ai/docs/introduction/what-is-portkey.md): Portkey AI is a comprehensive platform designed to streamline and enhance AI integration for developers and organizations. It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in your Generative AI apps.
- [Configure Logs Access Permissions for Workspace](https://docs.portkey.ai/docs/product/administration/configure-logs-access-permissions-in-workspace.md)
- [Enforce Budget Limits and Rate Limits for Your API Keys](https://docs.portkey.ai/docs/product/administration/enforce-budget-and-rate-limit.md): Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization
- [Enforcing Default Configs on API Keys](https://docs.portkey.ai/docs/product/administration/enforce-default-config.md): Learn how to attach default configs to API keys for enforcing governance controls across your organization
- [Enforcing Org Level Guardrails](https://docs.portkey.ai/docs/product/administration/enforce-orgnization-level-guardrails.md)
- [Enforcing Workspace Level Guardrails](https://docs.portkey.ai/docs/product/administration/enforce-workspace-level-guardials.md)
- [Enforcing Request Metadata](https://docs.portkey.ai/docs/product/administration/enforcing-request-metadata.md)
- [AI Gateway](https://docs.portkey.ai/docs/product/ai-gateway.md): The world's fastest AI Gateway with advanced routing & integrated Guardrails.
- [Automatic Retries](https://docs.portkey.ai/docs/product/ai-gateway/automatic-retries.md): LLM APIs often have inexplicable failures. With Portkey, you can rescue a substantial number of your requests with our in-built automatic retries feature. 
- [Batches](https://docs.portkey.ai/docs/product/ai-gateway/batches.md): Run batch inference with Portkey
- [Cache (Simple & Semantic)](https://docs.portkey.ai/docs/product/ai-gateway/cache-simple-and-semantic.md)
- [Canary Testing](https://docs.portkey.ai/docs/product/ai-gateway/canary-testing.md): You can use Portkey's AI gateway to also canary test new models or prompts in different environments. 
- [Conditional Routing](https://docs.portkey.ai/docs/product/ai-gateway/conditional-routing.md)
- [Configs](https://docs.portkey.ai/docs/product/ai-gateway/configs.md): This feature is available on all Portkey plans.
- [Fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks.md)
- [Files](https://docs.portkey.ai/docs/product/ai-gateway/files.md): Upload files to Portkey and reuse the content in your requests
- [Fine-tuning](https://docs.portkey.ai/docs/product/ai-gateway/fine-tuning.md): Run your fine-tuning jobs with Portkey Gateway
- [Load Balancing](https://docs.portkey.ai/docs/product/ai-gateway/load-balancing.md): Load Balancing feature efficiently distributes network traffic across multiple LLMs.
- [Multimodal Capabilities](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities.md)
- [Function Calling](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/function-calling.md): Portkey's AI Gateway supports function calling capabilities that many foundational model providers offer. In the API call you can describe functions and the model can choose to output text or this function name with parameters.
- [Image Generation](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/image-generation.md): Portkey's AI gateway supports image generation capabilities that many foundational model providers offer.
- [Speech-to-Text](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/speech-to-text.md): Portkey's AI gateway supports STT models like Whisper by OpenAI.
- [Text-to-Speech](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/text-to-speech.md): Portkey's AI gateway currently supports text-to-speech models on `OpenAI` and `Azure OpenAI`.
- [Vision](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/vision.md): Portkey's AI gateway supports vision models like GPT-4V by OpenAI, Gemini by Google and more.
- [Realtime API](https://docs.portkey.ai/docs/product/ai-gateway/realtime-api.md): Use OpenAI's Realtime API with logs, cost tracking, and more!
- [Request Timeouts](https://docs.portkey.ai/docs/product/ai-gateway/request-timeouts.md): Manage unpredictable LLM latencies effectively with Portkey's **Request Timeouts**.
- [Strict OpenAI Compliance](https://docs.portkey.ai/docs/product/ai-gateway/strict-open-ai-compliance.md)
- [Universal API](https://docs.portkey.ai/docs/product/ai-gateway/universal-api.md): Portkey's Universal API provides a consistent interface to integrate a wide range of modalities (text, vision, audio) and LLMs (hosted OR local) into your apps.
- [Virtual Keys](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys.md): Portkey's virtual key system allows you to securely store your LLM API keys in our vault, utilizing a unique virtual identifier to streamline API key management.
- [Connect Bedrock with Amazon Assumed Role](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role.md): How to create a virtual key for Bedrock using Amazon Assumed Role Authentication
- [Budget Limits](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/budget-limits.md): Budget Limits lets you set cost limits on virtual keys
- [Rate Limits](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/rate-limits.md): Set Rate Limts to your virtual keys
- [Autonomous Fine-tuning](https://docs.portkey.ai/docs/product/autonomous-fine-tuning.md): Automatically create, manage, and execute fine-tuning jobs for Large Language Models (LLMs) across multiple providers.
- [Enterprise Offering](https://docs.portkey.ai/docs/product/enterprise-offering.md)
- [Access Control Management](https://docs.portkey.ai/docs/product/enterprise-offering/access-control-management.md): With customizable user roles, API key management, and comprehensive audit logs, Portkey provides the flexibility and control needed to ensure secure collaboration & maintain a strong security posture
- [Audit Logs](https://docs.portkey.ai/docs/product/enterprise-offering/audit-logs.md): Track and monitor all administrative activities across your Portkey organization with comprehensive audit logging.
- [Budget Limits](https://docs.portkey.ai/docs/product/enterprise-offering/budget-limits.md)
- [AWS](https://docs.portkey.ai/docs/product/enterprise-offering/cloud-marketplace/aws.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software using AWS Marketplace.
- [Enterprise Components](https://docs.portkey.ai/docs/product/enterprise-offering/components.md)
- [KMS Integration](https://docs.portkey.ai/docs/product/enterprise-offering/kms.md): Customers can bring their own encryption keys to Portkey AI to encrypt data at storage.
- [Logs Export](https://docs.portkey.ai/docs/product/enterprise-offering/logs-export.md)
- [Org Management](https://docs.portkey.ai/docs/product/enterprise-offering/org-management.md): A high-level introduction to Portkey's organization management structure and key concepts.
- [API Keys (AuthN and AuthZ)](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/api-keys-authn-and-authz.md): Discover how Admin and Workspace API Keys are used to manage access and operations in Portkey.
- [JWT Authentication](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/jwt.md): Configure JWT-based authentication for your organization in Portkey
- [Organizations](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/organizations.md): Understand the role and features of Organizations, the highest level of abstraction in Portkey's structure.
- [Azure Entra](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/scim/azure-ad.md): Setup Azure Entra for SCIM provisioning with Portkey.
- [Okta](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/scim/okta.md): Set up Okta for SCIM provisioning with Portkey.
- [Overview](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/scim/scim.md): SCIM integration with Portkey.
- [SSO](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/sso.md): SSO support for enterprises
- [User Roles & Permissions](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/user-roles-and-permissions.md): Learn about the different user roles and their associated permissions within Organizations and Workspaces.
- [Workspaces](https://docs.portkey.ai/docs/product/enterprise-offering/org-management/workspaces.md): Explore Workspaces, the sub-organizational units that enable granular project and team management.
- [Private Cloud Deployments](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments.md)
- [Enterprise Architecture](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/architecture.md): Comprehensive guide to Portkey's hybrid deployment architecture for enterprises
- [AWS](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/aws.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software on AWS, tailored to meet the needs of large-scale, mission-critical applications.
- [Azure](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/azure.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software on Microsoft Azure, tailored to meet the needs of large-scale, mission-critical applications. It includes specific recommendations for component sizing, high availability, disaster recovery, and integration with monitoring systems.
- [Cloudflare Workers](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/cloudflare-workers.md)
- [F5 App Stack](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/f5-app-stack.md)
- [GCP](https://docs.portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/gcp.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software on Google Cloud Platform (GCP), tailored to meet the needs of large-scale, mission-critical applications.
- [Security @ Portkey](https://docs.portkey.ai/docs/product/enterprise-offering/security-portkey.md): Portkey AI provides a secure, reliable AI gateway for the seamless integration and management of large language models (LLMs).
- [Guardrails](https://docs.portkey.ai/docs/product/guardrails.md): Ship to production confidently with Portkey Guardrails on your requests & responses
- [Acuvity](https://docs.portkey.ai/docs/product/guardrails/acuvity.md): Acuvity is model agnostic GenAI security solution. It is built to secure existing and future GenAI models, apps, services, tools, plugins and more.
- [Aporia](https://docs.portkey.ai/docs/product/guardrails/aporia.md)
- [AWS Bedrock Guardrails](https://docs.portkey.ai/docs/product/guardrails/bedrock-guardrials.md): Secure your AI applications with AWS Bedrock's guardrail capabilities through Portkey.
- [Bring Your Own Guardrails](https://docs.portkey.ai/docs/product/guardrails/bring-your-own-guardrails.md): Integrate your custom guardrails with Portkey using webhooks
- [Creating Raw Guardrails (in JSON)](https://docs.portkey.ai/docs/product/guardrails/creating-raw-guardrails-in-json.md): With the raw Guardrails mode, we let you define your Guardrail checks & actions however you want, directly in code.
- [Guardrails for Embedding Requests](https://docs.portkey.ai/docs/product/guardrails/embedding-guardrails.md): Apply security and data validation measures to vector embedding requests to protect sensitive information and ensure data quality.
- [Lasso Security](https://docs.portkey.ai/docs/product/guardrails/lasso.md): Lasso Security protects your GenAI apps from data leaks, prompt injections, and other potential risks, keeping your systems safe and secure.
- [List of Guardrail Checks](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks.md)
- [Mistral](https://docs.portkey.ai/docs/product/guardrails/mistral.md): Mistral moderation service helps detect and filter harmful content across multiple policy dimensions to secure your AI applications.
- [Pangea](https://docs.portkey.ai/docs/product/guardrails/pangea.md): Pangea AI Guard helps analyze and redact text to prevent model manipulation and malicious content.
- [Patronus AI](https://docs.portkey.ai/docs/product/guardrails/patronus-ai.md): Patronus excels in industry-specific guardrails for RAG workflows.
- [PII Redaction](https://docs.portkey.ai/docs/product/guardrails/pii-redaction.md): Replace any sensitive data in requests with standard identifiers
- [Pillar](https://docs.portkey.ai/docs/product/guardrails/pillar.md)
- [MCP](https://docs.portkey.ai/docs/product/mcp.md)
- [Observability (OpenTelemetry)](https://docs.portkey.ai/docs/product/observability.md): Gain real-time insights, track key metrics, and streamline debugging with our comprehensive observability suite.
- [Analytics](https://docs.portkey.ai/docs/product/observability/analytics.md)
- [Auto-Instrumentation [BETA]](https://docs.portkey.ai/docs/product/observability/auto-instrumentation.md): Portkey's auto-instrumentation allows you to instrument tracing and logging for multiple LLM/Agent frameworks and view the logs, traces, and metrics in a single place.
- [Budget Limits](https://docs.portkey.ai/docs/product/observability/budget-limits.md)
- [Feedback](https://docs.portkey.ai/docs/product/observability/feedback.md): Portkey's Feedback APIs provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app.
- [Filters](https://docs.portkey.ai/docs/product/observability/filters.md)
- [Logs](https://docs.portkey.ai/docs/product/observability/logs.md): The Logs section presents a chronological list of all the requests processed through Portkey.
- [Logs Export](https://docs.portkey.ai/docs/product/observability/logs-export.md): Easily access your Portkey logs data for further analysis and reporting
- [Metadata](https://docs.portkey.ai/docs/product/observability/metadata.md): Add custom context to your AI requests for better observability and analytics
- [Tracing](https://docs.portkey.ai/docs/product/observability/traces.md): The **Tracing** capabilities in Portkey empowers you to monitor the lifecycle of your LLM requests in a unified, chronological view.
- [Open Source](https://docs.portkey.ai/docs/product/open-source.md)
- [Feature Comparison](https://docs.portkey.ai/docs/product/product-feature-comparison.md): Comparing Portkey's Open-source version and Dev, Pro, Enterprise plans.
- [Prompt Engineering Studio](https://docs.portkey.ai/docs/product/prompt-engineering-studio.md)
- [Prompt API](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-api.md): Learn how to integrate Portkey's prompt templates directly into your applications using the Prompt API
- [Guides](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-guides.md): Learn how to get the most out of Portkey Prompts with these practical guides
- [Integrations](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-integration.md)
- [Prompt Library](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-library.md)
- [Prompt Observability](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-observability.md)
- [Prompt Partials](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-partial.md): With Prompt Partials, you can save your commonly used templates (which could be your instruction set, data structure explanation, examples etc.) separately from your prompts and flexibly incorporate them wherever required.
- [Prompt Playground](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-playground.md)
- [Prompt Versioning & Labels](https://docs.portkey.ai/docs/product/prompt-engineering-studio/prompt-versioning.md)
- [Tool Library](https://docs.portkey.ai/docs/product/prompt-engineering-studio/tool-library.md)
- [PII Redaction](https://docs.portkey.ai/docs/product/security/pii.md)
- [Common Errors & Resolutions](https://docs.portkey.ai/docs/support/common-errors-and-resolutions.md): Since Portkey functions as a gateway - you may encounter Portkey-related, as well as non-Portkey related erros while using our services.
- [Contact Us](https://docs.portkey.ai/docs/support/contact-us.md)
- [Developer Forum](https://docs.portkey.ai/docs/support/developer-forum.md): Are you navigating the challenging journey of transitioning LLMs from prototype stages to full-scale production? You're not alone. As this frontier of technology continues to expand, the roadmap isn't always clear. Best practices, guidelines, and efficient methodologies are still on the horizon.
- [December '23 Migration](https://docs.portkey.ai/docs/support/portkeys-december-migration.md)