Local Search and Simulated Annealing › Example

Michael BrenndoerferOctober 29, 20264 min read
Machine Learning from ScratchAll chapters
Part 15 of 15Heuristic and Evolutionary Optimization
Chapter 90Local Search and Simulated Annealing › Example
Reading section 6 of 10 · Example · 4 minSection 6 of 10 · 4 min

Section 6 of 10 · 4 min read

Local Search and Simulated Annealing › ExampleSection view · Sections in this chapter6 / 10 ↓
Reading tools Definitions, links & notes

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Notes & comments

Let us work one reversal entirely by hand to see how the bookkeeping fits together. Start from the nearest-neighbor permutation

πNN=(1,7,8,3,∣6,5,4,2),\pi^{\text{NN}} = (1, 7, 8, 3, \mid 6, 5, 4, 2),

where the vertical bar separates the two vehicles. The current objective is the sum of the arc lengths on the two closed tours:

f(πNN)=(d01+d17+d78+d83+d30)+(d06+d65+d54+d42+d20).f(\pi^{\text{NN}}) = (d_{01} + d_{17} + d_{78} + d_{83} + d_{30}) + (d_{06} + d_{65} + d_{54} + d_{42} + d_{20}).

Substituting the coordinates (with xx and yy given in the array above) gives:

  • d01=(1−0)2+(3−0)2=10≈3.1623d_{01} = \sqrt{(1-0)^2 + (3-0)^2} = \sqrt{10} \approx 3.1623
  • d17=(3−1)2+(1−3)2=4+4≈2.8284d_{17} = \sqrt{(3-1)^2 + (1-3)^2} = \sqrt{4+4} \approx 2.8284
  • d78=(5−3)2+(3−1)2=4+4≈2.8284d_{78} = \sqrt{(5-3)^2 + (3-1)^2} = \sqrt{4+4} \approx 2.8284
  • d83=(4−5)2+(5−3)2=1+4≈2.2361d_{83} = \sqrt{(4-5)^2 + (5-3)^2} = \sqrt{1+4} \approx 2.2361
  • d30=(0−4)2+(0−5)2=16+25≈6.4031d_{30} = \sqrt{(0-4)^2 + (0-5)^2} = \sqrt{16+25} \approx 6.4031

so the first route contributes 17.458317.4583. For the second route:

  • d06=(6−0)2+(1−0)2=36+1≈6.0828d_{06} = \sqrt{(6-0)^2 + (1-0)^2} = \sqrt{36+1} \approx 6.0828
  • d65=(8−6)2+(3−1)2=4+4≈2.8284d_{65} = \sqrt{(8-6)^2 + (3-1)^2} = \sqrt{4+4} \approx 2.8284
  • d54=(7−8)2+(6−3)2=1+9≈3.1623d_{54} = \sqrt{(7-8)^2 + (6-3)^2} = \sqrt{1+9} \approx 3.1623
  • d42=(2−7)2+(6−6)2=5.0000d_{42} = \sqrt{(2-7)^2 + (6-6)^2} = 5.0000
  • d20=(0−2)2+(0−6)2=40≈6.3246d_{20} = \sqrt{(0-2)^2 + (0-6)^2} = \sqrt{40} \approx 6.3246

Carrying unrounded distances through the sums, the second route contributes 23.398023.3980 and the total is f(πNN)≈40.8563f(\pi^{\text{NN}}) \approx 40.8563. Summing the displayed rounded arc values instead can change the final reported decimal.

Now consider the reversal with i=1i = 1 and j=3j = 3. Reversing positions 2,3,42, 3, 4 of the permutation (that is, the segment {7,8,3}\{7, 8, 3\}) produces

π′=(1,3,8,7,∣6,5,4,2).\pi' = (1, 3, 8, 7, \mid 6, 5, 4, 2).

The reversal leaves the first customer, 1, fixed but changes the last customer of the first route from 3 to 7. Since distances are symmetric, reversing the internal edges keeps their lengths unchanged. The two changed boundary edges are 1→71\to7 replaced by 1→31\to3, and 3→03\to0 replaced by 7→07\to0. Therefore

Δ=d13+d70−d17−d30=13+10−8−41≈−2.4637.\Delta=d_{13}+d_{70}-d_{17}-d_{30} =\sqrt{13}+\sqrt{10}-\sqrt{8}-\sqrt{41} \approx-2.4637.

Here Δ\Delta is the new total distance minus the old total distance, and dabd_{ab} is the Euclidean distance between nodes aa and bb, including depot node 00. The negative sign means the reversal improves the tour. The two edges strictly inside the reversed segment retain their total length. The changed entering edge lengthens, while the changed return edge shortens by more, producing the net improvement. Checking every changed boundary edge is essential when a reversed segment ends at a route boundary.

PythonIn [18]
CollapseExpand
def reverse_at(perm, i, j):
    p = list(perm)
    p[i : j + 1] = p[i : j + 1][::-1]
    return tuple(p)


base = objective(nn_perm)
reverse_deltas = []
for i, j in [(1, 3), (2, 3), (0, 3), (3, 4)]:
    p = reverse_at(nn_perm, i, j)
    reverse_deltas.append((i, j, objective(p) - base, p))
ConsoleOut [19]
CollapseExpand
reversal (1,3): delta = -2.4637  perm = (1, 3, 8, 7, 6, 5, 4, 2)
reversal (2,3): delta = +0.7225  perm = (1, 7, 3, 8, 6, 5, 4, 2)
reversal (0,3): delta = +0.0000  perm = (3, 8, 7, 1, 6, 5, 4, 2)
reversal (3,4): delta = +1.6437  perm = (1, 7, 8, 6, 3, 5, 4, 2)
ConsoleOut [20]
CollapseExpand
Best single reversal so far: 38.3926 at (1, 3, 8, 7, 6, 5, 4, 2)
Full tour improvement from NN to exact optimum: 3.5004

The output confirms that reversing {7,8,3}\{7,8,3\} to {3,8,7}\{3,8,7\} lowers the length by about 2.4637. Other reversals can worsen it, as the printed positive delta illustrates. Every complete candidate score costs one objective evaluation. A full scan contains 28 such calls; initialization and restarts also consume the 300-call limit. Annealing spends that same limit on its initial score and proposals, while NN-seeded random search uses one nearest-neighbor score followed by 299 random permutations.

For the acceptance rule, take a hypothetical worsening move with Δ=+0.5\Delta = +0.5. At temperature T=1.0T = 1.0 the acceptance probability is exp⁡(−0.5)≈0.607\exp(-0.5) \approx 0.607, so even a small worsening is accepted most of the time. At temperature T=0.1T = 0.1 it is exp⁡(−5)≈0.0067\exp(-5) \approx 0.0067, so this +0.5+0.5 worsening move is rarely accepted. And at temperature T=0.01T = 0.01 it is exp⁡(−50)≈2×10−22\exp(-50) \approx 2 \times 10^{-22}, effectively zero. That exponential controls acceptance of worsening proposals; annealing also requires a proposal rule, a temperature schedule, state updates, and a stopping budget. In the mathematical acceptance rule, a larger temperature accepts a fixed worsening move more often, while any finite positive temperature leaves a positive probability. In floating-point code, sufficiently small exponentials can underflow to zero.

Comments

Add to the conversation

A question, a correction, or an example from your own work.

No comments yet. Be the first to share your thoughts!

Newsletter

Stay up to date

Get new articles, book updates, and notes on AI and technology.

Unsubscribe at any time. Privacy policy