The Confidence Checklist for LLMs in Production
Portkey CEO Rohit Agarwal shares practical tips from his own experience on crafting production-grade & reliable LLM systems.
Read more LLM reliability tips here.
Latest
Towards Reasoning in Large Language Models: A Survey - Summary
This paper provides a comprehensive overview of the current state of knowledge on reasoning in Large Language Models (LLMs), including techniques for improving and eliciting reasoning in these models, methods and benchmarks for evaluating reasoning abilities, findings and implications of previous r
Are We Really Making Much Progress in Text Classification? A Comparative Review - Summary
This paper reviews and compares methods for single-label and multi-label text classification, categorizing them into bag-of-words, sequence-based, graph-based, and hierarchical methods. The findings reveal that pre-trained language models outperform all recently proposed graph-based and hierarchy-b
Post Processing Recommender Systems with Knowledge Graphs for Recency, Popularity, and Diversity of Explanations - Summary
The paper proposes a re-ranking approach for explainable recommender systems using knowledge graphs to optimize for recency, popularity, and diversity of explanations. The approach is evaluated on two public datasets and shows an increase in explanation quality while preserving recommendation utili
SLiC-HF: Sequence Likelihood Calibration with Human Feedback - Summary
The paper presents a new approach called SLiC-HF that uses Sequence Likelihood Calibration with Human Feedback to improve language models. The approach is shown to be effective on the TL;DR summarization task and is a simpler and more computationally efficient alternative to Reinforcement Learning
Boosted Prompt Ensembles for Large Language Models -Summary
The paper proposes a prompt ensembling method for large language models called 'boosted prompting', which uses a small dataset to construct a set of few shot prompts that together comprise a boosted prompt ensemble. The few shot examples for each prompt are chosen in a stepwise fashion to be 'hard'
FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance - Summary
The paper discusses the cost associated with querying large language models (LLMs) and proposes FrugalGPT, a framework that uses LLM APIs to process natural language queries within a budget constraint. The framework uses prompt adaptation, LLM approximation, and LLM cascade to reduce the inference