Foundation Models Report (2021) › The Problem: A Field Without a Framework

The challenge wasn't just that new AI systems were emerging rapidly. The deeper problem was conceptual: research and policy communities lacked a shared framework for understanding what made these systems fundamentally different from previous approaches to AI. Traditional machine learning systems were built for specific tasks. You trained a model to classify images, translate between two languages, or recommend products. Each system had a narrow purpose, limited capabilities, and required extensive task-specific training data. The emerging large-scale systems behaved differently. They performed well across many tasks with minimal task-specific training. They demonstrated capabilities they weren't explicitly trained for. They seemed to capture general knowledge that transferred across domains.

This created confusion and fragmentation. Different research groups used different terminology. Some called them "large language models," pointing to their size and focus on text. Others used "pre-trained models," referring to the training methodology. Companies described them as "general-purpose AI" or "base models." The lack of consistent terminology made it difficult to discuss their implications coherently. When researchers presented findings about GPT-3's capabilities, it wasn't clear whether these findings applied to BERT, T5, or other large-scale systems. The field needed a unifying concept that captured what these systems shared despite their differences.

Policy discussions suffered from the same conceptual vacuum. Regulators and policymakers lacked frameworks for understanding how foundation models differed from traditional AI systems. Should they be regulated differently? What were the unique risks they posed? How should society govern access and assign accountability? Without a clear conceptual framework, policy discussions defaulted to familiar categories: privacy concerns, algorithmic bias, transparency. While important, these categories didn't capture what was new about foundation models.their scale, their broad capabilities, their centralization of power, their emergent behaviors.

The research community also struggled with questions about focus and priorities. Should researchers optimize for larger scale, better architectures, more diverse training data, or better alignment techniques? What were the most important risks to address: bias, misuse, environmental impact, concentration of power? Without a framework for organizing these questions, research efforts remained fragmented. The field needed a systematic analysis that could guide research directions and resource allocation, helping the community understand both what foundation models could do and what challenges they posed.