Differential Evolution

A simulation may return a cost for each proposed design without providing a usable gradient. If several basins contain good designs, repeatedly improving one starting point can miss a better region. Gradient Descent and Numerical Optimization gave us local updates, Local Search and Simulated Annealing introduced stochastic search, and Genetic Algorithms used a population of discrete candidates. We now build a population search directly for continuous decision variables.

Differential Evolution (DE) forms a perturbation from the difference between two population members, scales it by a factor $F$, and adds it to a third member. That difference supplies a direction and length from the current population geometry. It contains no objective values and is not a gradient estimate.

Step lengths depend on the population: widely separated donors can produce long differences, and a contracting population produces shorter differences. This self-scaling mechanism needs no explicit cooling schedule, but contraction can occur around a poor solution as well as a good one.

This chapter develops DE/rand/1/bin from differential mutation, binomial crossover, greedy one-to-one replacement, and box-bound handling. We compare it with uniform random search and bounded multistart local optimization on a small synthetic objective. Every method receives the same number of candidate evaluations, including initialization. That controls evaluation effort; it does not imply equal runtime or a universal ranking.