DALL·E 2 (2022) › Limitations and Challenges
History of Language AIAll chapters DALL·E 2 (2022) › Limitations and ChallengesOne section · Sections in this chapter5 / 7 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Despite its impressive capabilities, DALL·E 2 faced several significant limitations that researchers and users needed to address. The model sometimes struggled with precise spatial relationships, occasionally generating images where objects appeared in unexpected positions or with incorrect relative sizes. This limitation became particularly apparent with complex prompts requiring multiple objects in specific arrangements, such as "a red ball to the left of a blue cube."
The model could also produce unintended biases. This reflected patterns in its training data. Certain professions, activities, or attributes might be associated with particular demographics in ways that perpetuated stereotypes. While OpenAI implemented safety measures and content filtering, completely eliminating bias proved challenging given the model's training on internet-scale data containing societal biases. These issues highlighted the value of careful curation and bias mitigation in training large generative models.
Another limitation was the model's occasional misunderstanding of negation or complex logical relationships in prompts. While DALL·E 2 excelled at generating images matching positive descriptions, it could struggle with prompts like "a room without windows" or "an animal that is not a dog," sometimes producing images that included the very elements being negated. This revealed gaps in the model's understanding of compositional logic and semantic relationships.
Like other generative models, DALL·E 2 could produce artifacts or hallucinations: elements that appeared realistic but didn't correspond to the prompt or violated physical laws. Text within generated images often appeared garbled, faces could show subtle distortions, and some images contained physically impossible structures. These limitations meant that generated images required careful review before use in professional contexts.
The computational requirements for training and inference were substantial, limiting accessibility. Training DALL·E 2 required enormous computational resources and large datasets, while generating a single image consumed significant GPU time. These requirements made it difficult for individuals or smaller organizations to train their own models or even run inference locally, creating a dependency on cloud services and powerful hardware.
DALL·E 2's commercial availability through OpenAI's API also raised questions about control and access. Unlike open-source models, users couldn't modify the model, fine-tune it for specific domains, or audit its training process. This centralized approach, while ensuring safety measures, limited researchers' ability to experiment with variations or understand the model's inner workings fully.