Image Generation › Image Generation: Diffusion Models and Text-to-Image Synthesis

For most of computing history, creating images required human artists. Generating a novel photorealistic scene, a painting in a specific style, or an illustration from a text description was something only people could do. That changed dramatically in the early 2020s, when diffusion models transformed image synthesis from a niche research curiosity into a technology capable of producing images indistinguishable from photographs and paintings. Today, you can type a sentence and receive a detailed, coherent image in seconds. Understanding how this works, at the level of math and mechanism, is the goal of this chapter.

The story of image generation is a story about probability. Generating an image means sampling from a distribution over pixels. The challenge is that this distribution is astronomically high-dimensional (a $512 \times 512$ RGB image has nearly 800,000 dimensions), highly structured, and deeply non-Gaussian. Getting samples that look like photographs, not random noise, requires a model that has learned the complex correlations between pixels at every scale, from fine texture to global composition.

This chapter covers the full arc of modern image generation. We start with the historical context, then examine the mathematical foundations of diffusion: how adding and removing noise creates a tractable generative process. We then examine how language is connected to image synthesis, enabling text-to-image generation. We cover image editing techniques that build on generation, and we discuss how to evaluate the quality of generated images. By the end, you will understand how these systems work, why diffusion became the dominant paradigm, and what its real limitations are.