MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning - Summary

Arxiv URL: https://arxiv.org/abs/2205.00445

Authors: Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, Dor Muhlgay, Noam Rozen, Erez Schwartz, Gal Shachaf, Shai Shalev-Shwartz, Amnon Shashua, Moshe Tenenholtz

Summary:

The paper discusses the limitations of large language models (LMs) and proposes a neuro-symbolic architecture called the Modular Reasoning, Knowledge and Language (MRKL) system that combines LMs with external knowledge sources and discrete reasoning modules to overcome these limitations.

Key Insights & Learnings:

• Large language models have inherent limitations due to lack of access to current and proprietary information sources, and lack of reasoning capabilities.
• Traditional approaches to deploying LMs result in model explosion and loss of versatility.
• The MRKL system consists of an extendable set of modules, including neural and symbolic modules, and a router that routes inputs to the best module.
• The MRKL system offers safe fallback, robust extensibility, interpretability, up-to-date information, access to proprietary knowledge, and compositionality.
• The MRKL system has been implemented as Jurassic-X by AI21 Labs and is being piloted.

Terms Mentioned: neuro-symbolic architecture, large language models, external knowledge sources, discrete reasoning modules, Modular Reasoning, Knowledge and Language (MRKL) system, neural modules, symbolic modules, model explosion, versatility, safe fallback, robust extensibility, interpretability, up-to-date information, proprietary knowledge, compositionality, AI21 Labs, Jurassic-X

Technologies / Libraries Mentioned: BERT, GPT-3, Jurassic-1, PaLM