Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Variable Relationships › Correlation: Standardized Relationship Strength

Michael BrenndoerferJune 3, 20254 min read

Machine Learning from ScratchAll chapters

Part 1 of 9Statistics 101

Chapter 6Variable Relationships › Correlation: Standardized Relationship Strength

Reading section 4 of 11 · Correlation: Standardized Relationship Strength · 4 minSection 4 of 11 · 4 min

Section 4 of 11 · 4 min read

Variable Relationships › Correlation: Standardized Relationship StrengthOne section · Sections in this chapter4 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Correlation addresses the scale-dependence problem of covariance by standardizing the measure. The Pearson correlation coefficient, the most commonly used correlation measure, divides the covariance by the product of the standard deviations of both variables. This normalization ensures that correlation always falls between -1 and +1, regardless of the original units.

The population correlation coefficient is:

\rho_{XY} = \frac{\text{Cov}(X, Y)}{\sigma_X \sigma_Y}

Where:

$\sigma_X$ : Standard deviation of $X$
$\sigma_Y$ : Standard deviation of $Y$

The sample correlation coefficient is:

r_{XY} = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^n (x_i - \bar{x})^2} \sqrt{\sum_{i=1}^n (y_i - \bar{y})^2}}

Correlation values have clear interpretations. A correlation of $r = 1$ indicates a perfect positive linear relationship where all points fall exactly on a line with positive slope. A correlation of $r = -1$ indicates a perfect negative linear relationship. A correlation of $r = 0$ suggests no linear relationship, though non-linear relationships may still exist. In practice, correlations between 0.7 and 1.0 or -0.7 and -1.0 are generally considered strong, 0.3 to 0.7 (or -0.3 to -0.7) moderate, and below 0.3 weak, though these thresholds vary by field and context.

The following visualization demonstrates these different correlation patterns, showing how various relationships between variables manifest in scatter plots and their corresponding correlation coefficients.

Out[3]:

Visualization

Scatter plot showing strong positive correlation. — Strong positive correlation (r ≈ 0.95). The data points closely follow an upward-sloping linear pattern, indicating that as X increases, Y tends to increase proportionally. The tight clustering around an imaginary line demonstrates the strong linear relationship.

Scatter plot showing a weak positive sample correlation of 0.19. — Weak correlation (r ≈ 0.19). The data points are scattered without a strong linear pattern. Knowing the value of X provides little information about the likely value of Y.

Scatter plot showing strong negative correlation. — Strong negative correlation (r ≈ -0.95). The data points closely follow a downward-sloping linear pattern, indicating that as X increases, Y tends to decrease proportionally. This inverse relationship is captured by the negative correlation coefficient.

Correlation measures only linear relationships. Two variables can have zero correlation yet still be strongly related through a non-linear pattern such as a quadratic or sinusoidal relationship. This limitation makes visualization essential: plotting your data before relying solely on correlation coefficients helps identify non-linear patterns that correlation cannot detect. The following example demonstrates this limitation with a quadratic relationship where the correlation coefficient is near zero despite a clear pattern.

Out[4]:

Visualization

Scatter plot showing quadratic relationship with zero correlation. — Quadratic relationship with near-zero correlation (r ≈ 0.00). Despite the clear parabolic pattern, the Pearson correlation coefficient is close to zero because correlation only captures linear relationships. This demonstrates why visualization is essential: a correlation coefficient alone would incorrectly suggest no relationship exists between these variables.