Sign in Subscribe

Latest

LLMs In Prod: Day 1

LLMs In Prod: Day 1

This is Part 1 of our analysis diving into Portkey's year-end LLM production data. Over the next few days, we'll be unpacking insights from over 2 trillion tokens worth of production data to reveal how the AI ecosystem evolved in 2024. Today, we're starting with the most crucial aspect: who's

Real-time Guardrails vs Batch Evals: Understanding Safety Mechanisms in LLM Applications

In the world of Large Language Model (LLM) applications, ensuring quality, safety, and reliability requires different types of safety mechanisms at different stages. Two distinct approaches serve unique purposes: real-time guardrails and batch evaluations (evals). Let's understand how they differ and why both matter. Real-time Guardrails: Automated Runtime Protection Real-time

Build vs Buy - LLM Gateways

The use of Large Language Models (LLMs) has changed how enterprises approach AI and automation. LLM Gateways serve as essential infrastructure for managing, securing, and optimizing AI models, helping companies organize their AI workflows. But when faced with the choice of building an LLM Gateway from scratch or buying an

Prompt Injection Attacks in LLMs: What Are They and How to Prevent Them

In February 2023, a Stanford student exposed Bing Chat’s confidential system prompt through a simple text input, revealing the chatbot’s internal guidelines and behavioral constraints. It was one of the first prompt injection attacks highlighting a critical security loophole in Large Language Models (LLMs) – AI models powering everything

Deep Dive: OpenAI's o1 - The Dawn of Deliberate AI

Deep Dive: OpenAI's o1 - The Dawn of Deliberate AI

December 2024 marks a pivotal moment in AI development with the release of OpenAI's o1 system card, revealing an AI that thinks before it speaks. Let's explore how this changes everything..

How to scale AI apps - Lessons from building a billion-scale AI Gateway

How to scale AI apps - Lessons from building a billion-scale AI Gateway

Discover the journey of Portkey.ai in building a billion-scale AI Gateway. Learn key lessons on managing costs, optimizing performance, and ensuring accuracy while scaling generative AI applications.

Why Portkey is the right AI Gateway for you

Discover why Portkey's purpose-built AI Gateway fulfills the unique demands of AI infrastructure. From intelligent guardrails to cost optimization, explore how Portkey empowers teams to scale AI with confidence.