Sum of Squared Errors (SSE) › Mathematical Foundation

The Sum of Squared Errors is defined as:

$$SSE = \sum_{i=1}^n (y_i - \hat{y}_i)^2$$

Where:

• $y_i$: The actual (observed) value for the $i$-th observation
• $\hat{y}_i$: The predicted value for the $i$-th observation from our model
• $n$: The total number of observations
• The summation runs over all $n$ data points

Residuals: The difference $(y_i - \hat{y}_i)$ is called a residual - it represents how far off our prediction is from the actual value. A residual can be positive (prediction too low) or negative (prediction too high).

Squaring: We square each residual to:

• Eliminate the sign (make all differences positive)
• Penalize larger errors more heavily (squared errors grow quadratically)
• Ensure the metric is always non-negative

Summation: Adding up all squared residuals gives us the total error across all observations.

SSE is also known by several other names in different contexts, which can sometimes lead to confusion when reading different textbooks or research papers. While the underlying calculation remains the same, the terminology may shift depending on the field or the author's preference. Here are some of the most common alternative names for SSE:

• Sum of Squared Residuals (SSR): Emphasizes that we're measuring residuals
• Residual Sum of Squares (RSS): Another common notation
• Error Sum of Squares: Focuses on the error component

All refer to the same mathematical concept: $\sum_{i=1}^n (y_i - \hat{y}_i)^2$