GPT-4 (2023) › The Problem

GPT-3 and GPT-3.5 could answer one problem correctly and fail on a nearby variant. This sensitivity to wording and sampling made their outputs difficult to rely on without verification, especially in professional or educational settings.

Multi-step mathematics and logical deduction were common failure modes. Producing a plausible chain of text did not guarantee that every step was valid or that the final answer followed from the premises.

Earlier GPT interfaces accepted text but not images. That excluded tasks in which the prompt depended on a chart, diagram, photograph, or other visual source.

Safety and alignment presented ongoing challenges. While GPT-3.5 incorporated reinforcement learning from human feedback (RLHF) to align behavior with human preferences, the models still occasionally produced harmful outputs, refused reasonable requests, or provided incorrect information confidently. The balance between being helpful and avoiding harm was difficult to achieve, and models sometimes erred on both sides: refusing legitimate requests while occasionally producing problematic content.

GPT-3.5 also scored below GPT-4 on several standardized tests reported in the GPT-4 technical report. Exam scores offered a reproducible comparison, though they covered only selected tasks and prompting conditions.

The training process itself posed challenges. Scaling language models required massive computational resources, careful data curation, and sophisticated training techniques. Finding the right balance of data quality, model scale, and training objectives to produce both capable and safe models was an ongoing research challenge. The field needed better methods for training models that were simultaneously more capable, more reliable, and better aligned with human values.