Genetic Algorithms › Formula
Machine Learning from ScratchAll chapters Genetic Algorithms › FormulaSection view · Sections in this chapter4 / 10 ↓
Reading tools Definitions, links & notes
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The intuition behind a genetic algorithm is easy to state: keep several candidates alive, let good ones reproduce, and trust variation plus selection to improve the population. The formulas below make that intuition precise. They answer four questions in order: how a candidate is encoded and scored, how parents are chosen, how children are built without breaking the encoding, and how the next generation is chosen. Each piece exists for a reason, and the reasons are what make the algorithm more than a random search with extra steps.
Representation, fitness, and feasible solutions
We begin with the object being searched over. Because the decision is the order in which eight customers are visited, the natural genotype is a permutation. Let , let index the customers, and let node be the depot. A genotype is a permutation
where:
- : the permutation (genotype), a length-8 ordering of the customers
- : the customer visited at position of the permutation
- : the set of customer indices
- : the number of customers (), hence the length of every permutation
so the search space is with elements. This is already a strong constraint: every point in the search space is a valid ordering, and every ordering contains each customer exactly once. The algorithm never has to reason about missing or duplicated customers.
The permutation alone is not yet a pair of routes. The phenotype is produced by a decoder that cuts the permutation into two blocks of four and closes each block into a depot tour:
where:
- : the phenotype of permutation , a pair of closed depot tours
- : the customers assigned to the first vehicle, visited in that order
- : the customers assigned to the second vehicle, visited in that order
- : the depot that opens and closes each tour
The decoder is doing two jobs at once. It assigns the first four customers to one vehicle and the last four to the other, and it makes every permutation feasible with respect to capacity, because each vehicle receives exactly four customers with and . That is not an accident of this instance: every customer demand is , every vehicle capacity is , and the total demand equals the total capacity . The decoder enforces the only capacity split that can work.
Writing for the Euclidean distance between nodes and , the objective is the total length of both tours: the sum of the two closed depot tours produced by the decoder, in which each route begins at the depot, visits its four assigned customers in permutation order, and returns to the depot. The sum has ten terms in total: two depot-to-first-customer arcs, six customer-to-customer arcs, and two last-customer-to-depot arcs.
where:
- : the permutation (genotype) being scored, a length-8 ordering of the customers
- : the customer visited at position of the permutation
- : the vehicle index, running over the two vehicles and
- : the first position of vehicle 's block ( for vehicle 1, for vehicle 2)
- : the last position of vehicle 's block ( for vehicle 1, for vehicle 2)
- : the Euclidean distance between nodes and
- : the depot node
- : the summed length of both closed tours, the quantity minimised
Read the outer sum first: it runs over the two vehicles, and . Inside each term there are three pieces:
- the arc from the depot to the first customer of the route,
- the three arcs that connect the four customers of the route in order,
- the arc back to the depot.
The index arithmetic is just bookkeeping that starts vehicle 1 at position and vehicle 2 at position : for we get and the inner sum runs over ; for we get and the inner sum runs over . The inner sum connects consecutive customers within a block, and the two outer distance terms close the block to the depot at both ends, so each route has five arcs: three customer-to-customer arcs connecting the four customers, plus the depot-to-first-customer arc and the last-customer-to-depot arc. Across both routes that is ten arcs, which matches the ten distance terms in the formula. The objective is a pure routing length: no vehicle cost, no lateness term, no service-time term. Every one of the ten arcs that the decoder creates is charged exactly once, and no arc is charged twice.
We use unit travel speed, so travel time equals Euclidean distance. Feasibility is checked by simulating each route. With the service start at node , the arrival, and the window bounds, and the service time, the recursion is
where:
- : the service start time at node , i.e. arrival held until the window opens
- : the arrival time at node (the clock after traveling from the previous node)
- : the opening of the time window at node
- : the closing of the time window at node
- : the service time at node
- : the Euclidean travel distance from node to the next node
The recursion is subject to for every node including the return to the depot, and to , where is the demand at node and is the vehicle capacity. Here every customer demand is and every vehicle capacity is , and the total demand equals the total capacity , so each vehicle must carry exactly four customers: the decoder enforces that automatically. All windows are , and since every coordinate lies in no arc exceeds . A route has five arcs and serves four customers, each with service time . Starting from the depot, the service start at the fourth customer is at most , and the return to the depot takes one more arc of at most and one more service time of , giving . Every permutation is therefore feasible here. That is deliberate: the wide windows isolate search behaviour, and this instance does not test restrictive time-window handling.
Because we minimise, lower cost is better, and tournaments can compare costs without converting them into a separate fitness scale. For illustration, the positive transform maps costs 38 and 40 to and . The latter is percent smaller than the former. That transform compresses this particular cost difference; how concentrated proportional selection becomes depends on the full population. Our implementation instead selects the lowest raw cost in each sampled tournament.
On this symmetric-distance instance with deliberately wide windows, swapping the vehicles or reversing either tour preserves cost and feasibility. The decoded route sequences themselves change. We define a canonical signature by choosing the lexicographically smaller direction for each route and sorting the two resulting sequences. It identifies unordered pairs of undirected tours. Counting these signatures measures phenotype diversity; raw permutations and Hamming distance measure a different, genotype-level form of diversity that can also affect crossover.
Selection, crossover, mutation, and elitism
Selection is the pressure that turns variation into progress. Tournament selection draws distinct individuals uniformly at random from the population and returns the one with the lowest cost. It is simple, it needs no fitness scaling, and its selection pressure has a clean closed form.
For this rank calculation, assume distinct costs, or a fixed total tie order applied consistently when selecting a winner. In a population of size , rank is best and rank is worst. An individual of rank wins if it is drawn and no better individual (one of the individuals with smaller rank) is drawn with it. The number of -subsets that contain it and otherwise contain only the worse individuals is , out of equally likely subsets, so
where:
- : the population size
- : the tournament size (the number of individuals drawn without replacement)
- : the rank of the individual, with the best and the worst
- : the number of -subsets containing the individual and only worse individuals
- : the total number of equally likely -subsets
Under that strict ranking, for and the unique best individual is selected with probability , three times the uniform rate of , while the worst individual with rank , hence , has probability . Raising sharpens the pressure; is mild and always picks the leader. Summing the expression over returns by the hockey-stick identity, , so it is a proper distribution. The formula describes selection under the stated ranking. Our code resolves a cost tie by the order in which contenders were sampled, so it does not assign a fixed strict rank to tied individuals. The rank formula alone does not predict a diversity-loss rate.
The selection pressure implied by this formula can be read directly. The following computation evaluates the rank probabilities for several tournament sizes.
PythonIn [2]Expand
import math
import numpy as np
selection_n = 20
selection_ranks = np.arange(1, selection_n + 1)
selection_k_values = (2, 3, 5)
selection_probabilities = {
k: np.array(
[
math.comb(selection_n - r, k - 1) / math.comb(selection_n, k)
for r in selection_ranks
],
dtype=float,
)
for k in selection_k_values
}VisualizationOut [3]Expand

Order crossover (OX) builds the child from two cut positions. Fix cut positions and parents and , and index positions from to as in code:
- copy the block: for every ;
- let be the values already used, and list the values of in cyclic order starting just after the block, with indices taken modulo , discarding any value in ;
- fill the remaining positions in the same cyclic order, modulo , with that filtered list.
where:
- : the number of customers (), hence the length of each permutation
- : the inclusive cut positions bounding the copied block of the first parent
- : the first parent permutation, which supplies the copied block
- : the second parent permutation, which supplies the remaining values in cyclic order
- : the child permutation built by the rule
- : the set of values already placed in the child from the copied block
The child is a permutation. The block contributes distinct values; the filtered list contributes exactly the values of that are not in , in some order; the two position sets are disjoint and cover all positions. Nothing is duplicated and nothing is lost, so with no repair step. OX preserves the copied block and the cyclic relative order of the remaining values from . Relative order does not preserve every adjacency: filtering out a value creates new neighbours, and the decoder may assign a value to a different vehicle. Interior adjacencies of the copied block survive, but even that block can cross the vehicle split. Feasible crossover is not an improvement guarantee.
Mutation in permutation space is a swap. Choosing two distinct positions and exchanging their values always yields a valid permutation, and there are possible swaps. Within a single route a swap touches the arcs incident to both positions, at most four arcs (three if the positions are adjacent). Swapping one position in the first block with one in the second block instead re-assigns two customers between vehicles, each touching its own incident arcs, so the change in objective can be much larger: several long arcs can change at once. Mutation is therefore a source of novelty, not a guarantee of improvement.
Elitism fixes the survival rule. With parents and offspring per generation, selection sets
Here and are indexed collections, and the union denotes concatenation with multiplicity: duplicate individuals still occupy separate slots. The lowest-cost slots survive. Assume the objective is fixed and deterministic. In detail:
- : the population of parents at generation
- : the offspring produced at generation
- : the number of parents retained each generation (the population size)
- : the operator that keeps the lowest-cost individuals of the union. Because the old population is included among the candidates, the lowest retained cost is no larger than the old minimum, so the incumbent satisfies : with elitism the best-so-far curve is non-increasing, and we record the incumbent after each evaluation in a separate archive. A running-minimum archive can provide such a trace under other survival rules too. The population may churn, but the best solution found so far never gets worse.
Finally, the budget identity. With a population of , the initial population costs evaluations (the shared nearest-neighbour score plus random members), and each completed generation costs evaluations. The total is:
where:
- : the total evaluation budget (here )
- : the single complete-route evaluation of the nearest-neighbour construction
- : the random permutations that fill the rest of the initial population
- : the population size, also the number of children charged per generation
- : the number of generations
The displayed identity is for complete generations. In general with , and the implementation can spend the remainder on a partial final generation. With and , and this gives generations. This identity is the accounting contract for the whole chapter: initialization, every child, and every duplicate are charged, so a comparison with the other methods is a comparison of equal evaluation budgets rather than equal wall-clock time.
The budget identity also makes the population-size trade-off explicit. The following computation tabulates the number of complete generations available for a range of population sizes at the same budget.
PythonIn [4]Expand
import numpy as np
budget_for_generations = 300
population_sizes = np.arange(5, 51, 5)
generations = np.floor(
(budget_for_generations - population_sizes) / population_sizes
).astype(int)VisualizationOut [5]Expand

Population diversity, repair, and premature convergence
Define the diversity of a generation as the number of distinct canonical signatures in the population, , with , where:
- : the number of distinct canonical signatures in generation
- : the canonical signature of permutation (vehicle-symmetric, reversal-invariant)
- : the population of parents at generation
- : the number of parents (population size)
For populations with , genotype-level diversity can also be measured with the mean pairwise Hamming distance , where:
- : the -th individual permutation in the population
- : a position index,
- : the number of parents (population size)
The sum runs over ordered pairs , so each unordered pair is counted twice, matching the denominator. This measures genotype difference: swapping the two halves changes Hamming distance while preserving the unordered tour pair. The signature count instead measures distinct phenotypes under the stated symmetry; neither measure fully characterizes what crossover can produce.
Diversity is what makes crossover useful, and this can be shown rather than asserted. Order crossover applied to identical parents returns the parent: the block matches, and the filtered list from (the values of not copied into the block, in the cyclic order in which they appear in ) is placed into the empty positions in cyclic order; the two cyclic orderings agree, so each value is placed back where it started. So if the population collapses to copies of one permutation, for every selected pair, and the only source of novelty is mutation. A population in that state is a single point with random kicks, which is precisely the trajectory we were trying to escape.
This loss of useful variation is one form of premature convergence. Selection can remove variants; mutation proposes new variants but does not guarantee their survival. A low mutation rate may leave identical parents reproducing copies. A high rate may perturb useful structures. The appropriate balance depends on the encoding, objective, survivor rule, and evaluation budget.
The balance is problem specific, and with a budget of only generations there is little room to recover from a bad setting mid-run. We illustrate the trade-off with actual runs in the visualisation section rather than claiming a rule of thumb.
Repair is one alternative when feasibility is not enforced by construction. A repair operator maps an infeasible genotype back into the feasible set, and a penalty method folds violations into the objective:
where:
- : the repair operator, which takes any genotype and returns a feasible one
- : the routing length of permutation , as defined above
- : a nonnegative violation measure, larger when capacity or windows are exceeded
- : the penalty weight that trades violation against routing length
The penalty objective compares violation and route cost on a chosen scale. A finite penalty need not make its minimizer feasible: a low route cost can outweigh the violation penalty. A validated repair may require computation or additional scoring, and many-to-one repair can reduce variation. Increasing amplifies violation differences rather than generally flattening the objective. Our decoder needs no repair on this wide-window instance. With restrictive windows, feasibility would need a suitable decoder, feasible moves, rejection, repair, penalties with explicit validation, or another constraint-handling rule.
Comments
Add to the conversation
A question, a correction, or an example from your own work.
No comments yet. Be the first to share your thoughts!