Multimodal Training Data › Limitations and Impact

The construction of multimodal training data faces basic tensions between scale and quality, automation and accuracy, that shape model capabilities and limitations in significant ways. These tensions are not merely technical inconveniences; they determine whether models develop real understanding or mere statistical correlations, whether they serve diverse global populations or only wealthy Western users, and whether they can be trusted for necessary applications.

Image-text pairs teach correlation, not causation or physical understanding. A model trained on "person holding umbrella" captions learns to associate umbrellas with rain, but not that umbrellas cause dryness or that rain falls downward. This produces models that excel at retrieval and captioning yet struggle with physical reasoning or counterfactual scenarios ("what if the umbrella were inside out?").

Interleaved data mitigates this by giving contextual reasoning chains, but the underlying supervision remains correlational. The model learns that certain visual patterns co-occur with certain linguistic patterns, not the physical, causal, or social relationships that connect them. This limitation becomes apparent when models confidently describe impossible scenes or fail to recognize physical contradictions in generated images. A model asked whether a photograph shows physically plausible bridge construction may hallucinate an affirmative answer if the image superficially resembles training examples where such captions were common.

The gap between correlation and causation is not unique to multimodal models: it affects all statistical learning systems. What makes multimodal training data particularly vulnerable is the diversity of ways that text and images can be spuriously correlated on the web. Photographs of luxury goods often appear with aspirational captions that do not describe the image but rather the desired emotional response. News photographs often appear with captions that describe context invisible in the image. Art photographs often appear with abstract descriptions that resist literal visual interpretation. Each of these creates correlations that the model must somehow learn to discount.

The large majority of high-quality image-text pairs feature English captions. While Part XXV: Encoder-Decoder Models discussed multilingual training for text-only models, multimodal models face a steeper challenge. Visual concepts may be culture-specific, such as different hand gestures, architectural styles, or food presentations. Non-English captions are often lower quality or direct translations that lose culturally-specific visual nuance.

This limits transfer to low-resource languages and embeds Western visual assumptions as universal defaults. A model trained primarily on English captions may fail to recognize culturally specific practices, clothing, or environments, effectively rendering invisible the visual experiences of billions of people. Even when non-English text is included, the association between visual and linguistic representations may be weaker for non-English languages because fewer training examples reinforce those connections. The model may default to English-language descriptions when prompted in other languages, even when the image contains culturally specific content that has natural names in the target language.

Addressing this requires intentional curation of non-English, non-Western visual content with high-quality native language captions. Some research groups have created targeted multilingual datasets that explicitly pair images from specific cultural contexts with captions written by native speakers rather than translated from English. These efforts are promising but face the basic challenge of scale: creating billions of high-quality multilingual image-text pairs requires an enormous sustained human effort.

Vision-language models memorize training examples. When benchmark images appear in pre-training data, models reach inflated performance by retrieval rather than reasoning. Unlike text, where n-gram overlap detection can identify contamination, image deduplication requires perceptual hashing that misses semantic duplicates (the same scene photographed from slightly different angles).

As models train on increasingly large web-scraped datasets, the probability that evaluation benchmarks are "leaked" into training approaches certainty, complicating the measurement of true generalization that we will explore in upcoming chapters on evaluation. This is particularly concerning for safety-necessary applications where we need to know whether the model understands concepts or merely recognizes specific test instances.

The memorization problem also raises privacy concerns. Models trained on web-scraped data may memorize specific individuals' photographs, letting identification of private individuals from generated or described content. This is analogous to the text memorization problem, where language models can reproduce training text verbatim, but with the added concern that visual memorization involves personal images that individuals may not have consented to include in training data.

Aggressive filtering improves model quality but reduces diversity and scale. LAION-5B filtered to high-aesthetic, high-alignment subsets retains only 10-20% of original data. This creates a tension: Chinchilla scaling laws from Part XXIII: Scaling Laws suggest models need large data, yet quality constraints limit available quantity.

The result is that compute-optimal training often becomes data-constrained for multimodal models, driving interest in synthetic data generation despite its compounding bias risks. Researchers must choose between training on large amounts of noisy data (risking learning spurious correlations) or small amounts of clean data (risking poor generalization to rare concepts). Neither option is clearly superior: the right balance depends on the target application, the available compute budget, and the specific failure modes that are most consequential.

One promising direction is staged training with different data mixtures at different stages. Early training uses large, noisy datasets to establish broad visual and linguistic coverage. Later training uses smaller, higher-quality datasets to refine representations and reduce noise-induced biases. This mirrors the two-stage approach used in text pre-training, where broad pre-training is followed by targeted domain-specific fine-tuning.

Training data shows and amplifies societal biases. Image-text pairs associate certain professions with specific genders or ethnicities due to photography distribution on the web. Stock photo sites, which contribute heavily to web-scraped datasets, often perpetuate stereotypical representations. When these models generate captions or answer questions about images, they perpetuate these stereotypes, potentially causing real-world harm by reinforcing biased associations.

Unlike text-only models where bias can be mitigated through careful corpus curation or RLHF as covered in Part XXXVII: Alignment and RLHF, multimodal biases are harder to detect because they require visual inspection of the training data, which is computationally expensive at billion-image scale. Auditing for demographic bias requires processing images through detection pipelines that may themselves be biased, creating a methodological challenge for fairness research.

The practical implication is that multimodal models should not be treated as neutral visual analysts. They bring the biases of their training data to every analysis, and users who rely on them for consequential decisions (medical diagnosis, security screening, recruitment) may inadvertently inherit and amplify those biases. Responsible deployment requires both technical mitigation strategies and organizational awareness that model outputs are shaped by data construction choices made during the training pipeline.