Foundation Models Report (2021) › 2021: Foundation Models Report

By 2021, several general-purpose model families had emerged. GPT-3 could generate coherent essays, write code, translate languages, and solve complex reasoning problems with minimal task-specific training. BERT and its variants achieved state-of-the-art performance across dozens of natural language understanding benchmarks. Vision transformers showed that the same architectural principles could work across modalities. These systems were powerful, but what exactly were they? How should researchers and practitioners think about them? What principles should guide their development and deployment?

The field lacked a unifying framework. Researchers used different terminology: "large language models," "pre-trained models," "general-purpose models," "base models." Companies developed systems in isolation, each trained and built differently and offering different capabilities. There was no shared vocabulary for discussing the opportunities and risks these systems presented. Policy discussions lacked a conceptual framework for understanding what made these systems different from previous AI approaches. The community needed a way to organize thinking about this new class of systems and their implications.

In August 2021, a team of over 100 researchers at Stanford's Center for Research on Foundation Models (CRFM), led by Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, and many others, published "On the Opportunities and Risks of Foundation Models." The report filled the conceptual gap by formally defining and analyzing "foundation models" as a new paradigm in artificial intelligence. It provided the first systematic examination of models trained on broad data at scale that could be adapted to diverse downstream tasks. Beyond the technical analysis, it examined policy questions alongside social and ethical implications. This offered a framework that would shape discourse and research agendas across the AI community for years to come.

The Foundation Models Report represented a watershed moment precisely because it provided the common language and conceptual framework the field desperately needed. It established foundation models as a distinct category worthy of study in their own right, not just as scaled-up versions of existing approaches. The report's analysis of opportunities and risks influenced policy discussions, research directions, and the development of new AI systems, making it one of the most influential documents in the recent history of artificial intelligence. Its framework continues to structure how we understand and discuss large-scale AI systems today.