Loss Functions › Loss Functions

Every neural network needs a way to measure how wrong its predictions are. Without that signal, training is impossible. The loss function is that measurement, a scalar value that tells the optimizer whether the model is getting closer to correct predictions or wandering further away.

You have already seen how linear classifiers produce scores, how activation functions introduce non-linearity, and how MLPs stack layers to represent complex functions. But none of that training happens without a loss function. The loss function is the bridge between the model's output and the training signal that flows backward through backpropagation. Choosing the right loss function is not a minor implementation detail. It shapes what the model optimizes, what errors it penalizes most, and whether it even converges.

This chapter covers the most important loss functions for neural networks and NLP: regression losses, classification losses, and more specialized objectives for tasks like contrastive learning and class imbalance. For each one, you will understand why it was designed that way, what it measures, and when to choose it over alternatives.