Normalization › Practical Implementation
Machine Learning from ScratchAll chapters Normalization › Practical ImplementationOne section · Sections in this chapter5 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Proper Train-Test Split with Normalization
This example demonstrates the correct way to apply normalization in a machine learning pipeline:
import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor
from sklearn.preprocessing import MinMaxScaler
# Set random seed for reproducibility
np.random.seed(42)
# Create sample dataset with different scales
X = np.random.randn(100, 3)
X[:, 0] *= 1000 # Scale first feature to be much larger
X[:, 1] *= 10 # Scale second feature moderately
# Third feature remains small scale
# Create target variable
y = 2 * X[:, 0] + 3 * X[:, 1] + 0.5 * X[:, 2] + np.random.randn(100) * 0.1
# Split data
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
# CORRECT: Fit scaler only on training data
scaler = MinMaxScaler()
X_train_normalized = scaler.fit_transform(X_train)
X_test_normalized = scaler.transform(X_test)
# Train model
model = MLPRegressor(hidden_layer_sizes=(50,), max_iter=1000, random_state=42)
model.fit(X_train_normalized, y_train)
# Make predictions
y_pred = model.predict(X_test_normalized)
mse = mean_squared_error(y_test, y_pred)
print("Model performance:")
print("Test MSE:", mse)
print("Training data min:", np.min(X_train_normalized, axis=0))
print("Training data max:", np.max(X_train_normalized, axis=0))Model performance: Test MSE: 2116358.4104419583 Training data min: [0. 0. 0.] Training data max: [1. 1. 1.]
Key Implementation Guidelines
Normalization is a simple but important step in the machine learning workflow. Here are the main guidelines to follow:
-
Fit the scaler only on the training data.
This ensures that information from the test set does not leak into the model during training. Fitting on the entire dataset (including the test set) can lead to overly optimistic performance estimates and poor generalization. -
Transform both training and test data using the same scaler.
After fitting the scaler on the training data, use it to transform both the training and test sets. This guarantees that the scaling parameters (min and max values) are consistent and based solely on the training data. -
Never fit the scaler on the entire dataset before splitting.
Doing so introduces data leakage, as the test set statistics influence the scaling of the training data. -
Use pipelines to automate and safeguard the process.
Scikit-learn pipelines help ensure that normalization and modeling steps are applied correctly and in the right order, reducing the risk of data leakage and making your workflow more reproducible.
By following these guidelines, you ensure that your model evaluation is fair and that your results will generalize well to new, unseen data.
from sklearn.pipeline import Pipeline
# Create pipeline with normalization and model
pipeline = Pipeline(
[
("scaler", MinMaxScaler()),
(
"model",
MLPRegressor(
hidden_layer_sizes=(50,), max_iter=1000, random_state=42
),
),
]
)
# Fit pipeline on training data
pipeline.fit(X_train, y_train)
# Make predictions on test data
y_pred_pipeline = pipeline.predict(X_test)
mse_pipeline = mean_squared_error(y_test, y_pred_pipeline)
print("Pipeline MSE:", mse_pipeline)Pipeline MSE: 2116358.4104419583