Multilayer Perceptrons › MLP for Regression
Michael BrenndoerferApril 29, 20253 min read
Language AI HandbookAll chapters Multilayer Perceptrons › MLP for RegressionSection view · Sections in this chapter8 / 13 ↓
Sections in this chapter13
Reading tools Definitions, links & notes
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Notes & comments
Regression tasks require predicting continuous values rather than class labels. The key differences from classification are:
- No activation function on the output layer (identity function, allowing unbounded predictions)
- Mean squared error (MSE) or mean absolute error (MAE) as the loss function
- Output layer has one neuron per target variable
PythonIn [30]Expand
# Create a regression dataset: sin(x) + 0.1 * x^2 with Gaussian noise
np.random.seed(42)
X_reg = np.random.uniform(-3, 3, (500, 1))
y_reg = np.sin(X_reg) + 0.1 * X_reg**2 + np.random.normal(0, 0.15, X_reg.shape)
X_train_reg, X_test_reg, y_train_reg, y_test_reg = train_test_split(
X_reg, y_reg, test_size=0.2, random_state=42
)
X_train_reg_t = torch.FloatTensor(X_train_reg)
y_train_reg_t = torch.FloatTensor(y_train_reg)
X_test_reg_t = torch.FloatTensor(X_test_reg)
y_test_reg_t = torch.FloatTensor(y_test_reg)PythonIn [31]Expand
class RegressionMLP(nn.Module):
def __init__(self, input_size, hidden_sizes):
super().__init__()
layers = []
prev_size = input_size
for hidden_size in hidden_sizes:
layers.append(nn.Linear(prev_size, hidden_size))
layers.append(nn.ReLU())
prev_size = hidden_size
# Output layer with no activation for regression
layers.append(nn.Linear(prev_size, 1))
self.network = nn.Sequential(*layers)
def forward(self, x):
return self.network(x)
model_reg = RegressionMLP(input_size=1, hidden_sizes=[32, 16])
criterion_reg = nn.MSELoss()
optimizer_reg = optim.Adam(model_reg.parameters(), lr=0.01)
train_losses_reg = []
for epoch in range(300):
model_reg.train()
y_pred_reg = model_reg(X_train_reg_t)
loss = criterion_reg(y_pred_reg, y_train_reg_t)
optimizer_reg.zero_grad()
loss.backward()
optimizer_reg.step()
train_losses_reg.append(loss.item())VisualizationsOut [32]Expand


ConsoleOut [33]Expand
Regression Results: Test MSE: 0.0245 Test RMSE: 0.1564 Training Progress: Initial MSE: 0.8912 Final MSE: 0.0220 Improvement: 97.5%
The low test MSE and RMSE indicate the model fits the underlying function well. Unlike polynomial regression where you must choose the degree up front, the MLP automatically discovers the appropriate level of complexity through its hidden representations.
Comments
Add to the conversation
A question, a correction, or an example from your own work.
1 comment
fantastic article
Thank you very much for your comment!