Multilayer Perceptrons and Backpropagation › Advantages

The main advantage of an MLP is representational flexibility. A hidden layer applies several learned affine projections followed by a nonlinearity, so the network can construct features that are not present in the raw input coordinates. In a two-dimensional classification problem, this lets the decision boundary bend. In higher-dimensional tabular problems, it lets the model represent interactions such as "high income matters only when the account is new." Logistic regression on the original features has additive log-odds; to represent an income-by-account-age interaction in the log-odds, it needs additional interaction or nonlinear features.

The same hidden-layer machinery extends to several prediction tasks. A sigmoid output with binary cross-entropy gives binary classification; a softmax output with cross-entropy gives multiclass classification; a linear output with squared error gives regression. The output dimension is chosen for the target representation: one logit for binary classification, one logit per class for a softmax output, and one value per regression target. The output activation and loss must match that task. The hidden-layer chain-rule calculation follows the same principle.

The forward and backward passes can be expressed with matrix multiplications and elementwise operations, with examples arranged in batches. Mini-batch training uses fewer examples for each update than full-batch gradient descent and averages more per-example gradients than one-example stochastic gradient descent. Regularization can be added directly to the loss, and validation data can monitor the resulting training loop.