Portkey Blog (Page 2)

Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference - Summary

The research paper introduces ModernBERT, an updated version of the original BERT model, which is an encoder-only transformer designed to improve retrieval and classification tasks. Despite the original BERT's widespread use, until now, there have been limited improvements in te

What is tree of thought prompting?

Large language models (LLMs) keep getting better, and so do the ways we work with them. Tree of thought prompting is a new technique that helps LLMs solve complex problems. It works by breaking down the model's thinking into clear steps, similar to how humans work through difficult problems. This

Prompt engineering techniques for effective AI outputs

Remember when prompt engineering meant just asking ChatGPT to write your blog posts or answer a basic question? Those days are long gone. We're seeing companies hire dedicated prompt engineers now - it's become a real skill in getting large language models (LLMs) to do exactly what you need them

Top 10 MCP Servers

Anthropic recently launched Model Context Protocol - a standardized protocol that governs how models can interact with local and remote sources Here's a list of production-ready and experimental MCP servers you can access for database connections, cloud, and infra, Content, productivity tools, etc. Data and storage MCP server 1. PostgreSQL

Model Context Protocol for building reliable, enterprise LLM applications

Picture the modern enterprise LLM application scene - from customer service chatbots parsing thousands of queries to data analysis systems processing vast business insights. Large Language Models (LLMs) power these systems, but there's a critical challenge that many organizations overlook: context management. As enterprises scale their LLM deployments, they face

LLMs in Prod: The Reality of AI Outages, No LLM is Immune

This is Part 2 of our series analyzing Portkey's critical insights from production LLM deployments. Today, we're diving deep into provider reliability data from 650+ organizations, examining outages, error rates, and the real impact of downtime on AI applications. From the infamous OpenAI outage to the daily challenges of rate

Latest