Open Sourcing Guardrails on the Gateway Framework
We are solving the *biggest missing component* in taking AI apps to prod → Now, enforce LLM behavior and route requests with precision, in one go.
Topic
Production Guides
A collection of 33 issues
Portkey & Patronus - Bringing Responsible LLMs in Production
Patronus AI's suite of evaluators are now available on the Portkey Gateway.
Supercharging Open-source LLMs: Your Gateway to 250+ Models
The Rise of Open-source LLMs in Production
⭐️ Implementing FrugalGPT: Reducing LLM Costs & Improving Performance
FrugalGPT is a framework proposed by Lingjiao Chen, Matei Zaharia, and James Zou from Stanford University in their 2023 paper "FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance". The paper outlines strategies for more cost-effective and performant usage of large language model (LLM)
What It Means To Go To Prod
Lately, I am feeling more and more confident about Portkey solving *real* production challenges for our users.
Of course, developer love and revenue increase are important parameters, but there's something else just as high signal:
The platform modularity that customers demand from Portkey, and how fast we serve
Portkey Goes Multimodal
2024 is the year where Gen AI innovation and productisation happens hand-in-hand.
We are seeing companies and enterprises move their Gen AI prototypes to production at a breathtaking pace. At the same time, an exciting new shift is also taking place in how you can interact with LLMs: completely new
Understanding RAG: A Deeper Dive into the Fusion of Retrieval and Generation
Retrieval-Augmented Generation (RAG) models represent a fascinating marriage of two distinct but complementary components: retrieval systems and generative models. By seamlessly integrating the retrieval of relevant information with the generation of contextually appropriate responses, RAG models achieve a level of sophistication that sets them apart in the realm of artificial
⭐️ OpenAI Model Deprecation Guide
In two days (i.e. Jan 4), OpenAI will retire 33 models, including GPT-3 (text-davinci-003) and various others. This is OpenAI's biggest model deprecation so far. Here's what you need to know:
GPT-3 Model Retirement
The text-davinci-003 model (commonly known as GPT-3) will be unavailable from
Anyscale's OSS Models + Portkey's Ops Stack
The landscape of AI development is rapidly evolving, and open-source Large Language Models (LLMs) have emerged as a key foundation for building AI applications.
Anyscale has been a game-changer here with their fast and cheap APIs for Llama2, Mistral, and more OSS models.
But to harness the full potential of
⭐ Building Reliable LLM Apps: 5 Things To Know
In this blog post, we explore a roadmap for building reliable large language model applications. Let’s get started!