Normalization › Mathematical Foundation

Michael BrenndoerferMay 19, 20252 min read
Machine Learning from ScratchAll chapters
Part 3 of 9Foundations
Chapter 26Normalization › Mathematical Foundation
Reading section 3 of 10 · Mathematical Foundation · 2 minSection 3 of 10 · 2 min

Book contents

Section 3 of 10 · 2 min read

Normalization › Mathematical FoundationOne section · Sections in this chapter3 / 10 ↓
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The Min-Max Normalization Formula

For a feature jj with nn observations, min-max normalization transforms each value xijx_{ij} to xij′x'_{ij} using:

xij′=xij−min⁡(xj)max⁡(xj)−min⁡(xj)x'_{ij} = \frac{x_{ij} - \min(x_j)}{\max(x_j) - \min(x_j)}

where:

  • xij′x'_{ij} is the normalized value of feature jj for observation ii
  • min⁡(xj)\min(x_j) is the minimum value of feature jj
  • max⁡(xj)\max(x_j) is the maximum value of feature jj

Key Properties

After normalization, each feature in your dataset is transformed so that it has a minimum value of 0 and a maximum value of 1. This transformation ensures that all features, regardless of their original range or units, are directly comparable and contribute equally to the analysis. In practical terms, this means that no single feature will dominate the learning process simply because it has a larger numeric range. Instead, every feature is scaled to the same [0, 1] interval, allowing algorithms.especially those sensitive to feature magnitude, such as neural networks or distance-based methods.to perform optimally and fairly.

  • Minimum: min⁡(xj′)=0\min(x'_j) = 0 for all features
  • Maximum: max⁡(xj′)=1\max(x'_j) = 1 for all features
  • Range: Range(xj′)=1\text{Range}(x'_j) = 1 for all features

This ensures that all features contribute equally to distance calculations and optimization processes.