Descriptive Statistics › Measures of Distribution Shape
Machine Learning from ScratchAll chapters Descriptive Statistics › Measures of Distribution ShapeOne section · Sections in this chapter5 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Beyond center and spread, distribution shape gives useful information about data behavior. Two common measures, skewness and kurtosis, summarize asymmetry and tail weight, respectively [9].
Skewness
One common sample measure of skewness uses central moments computed with a consistent denominator:
For nonconstant data, and the statistic is defined and unitless. Some software instead reports a finite-sample bias-adjusted estimator, so comparisons should use the same convention.
A nondegenerate symmetric distribution with a finite third moment has skewness zero, but zero skewness does not by itself prove symmetry. Positive skewness, or right skew, often accompanies a longer or heavier right tail. In many familiar unimodal cases the mean then exceeds the median, but that ordering is not guaranteed for every distribution.
Negative skewness, or left skew, often accompanies a longer or heavier left tail. Test scores on an easy exam can show this pattern when many students score near the upper boundary and a smaller number receive much lower scores.
There is no universal cutoff that turns a skewness value into “moderate” or “high.” Interpret its magnitude relative to the estimator, sample size, application, and the accompanying distribution plot.
Kurtosis
Moment kurtosis standardizes the fourth central moment:
For nonconstant data, and is defined. Because fourth powers amplify large standardized deviations, kurtosis is sensitive to extreme observations. It is not a direct partition of probability between the center and tails, nor does it determine whether a density has a sharp or flat peak [10]. A normal distribution has population kurtosis 3. Many software packages instead report excess kurtosis, , and may also apply a finite-sample bias correction; under the excess convention the normal reference is zero.
High kurtosis means that extreme standardized deviations contribute strongly to the fourth moment. It can be a warning that a normal model understates important extremes, but it does not guarantee a larger exceedance probability at every threshold. Risk analysis should therefore examine relevant tail probabilities and quantiles directly rather than rely on kurtosis alone.
Low kurtosis means that extreme standardized deviations make a smaller fourth-moment contribution than under the chosen reference. A continuous uniform distribution, for example, has raw kurtosis 1.8. This value does not by itself characterize the height or flatness of its density peak.
Kurtosis is one diagnostic for departures from a normal model, not a method-selection rule. When extremes matter, inspect distribution plots, tail quantiles, and model residuals and then choose methods whose assumptions fit the task.
You might wonder why the third and fourth powers in the skewness and kurtosis formulas capture asymmetry and tail behavior, respectively. The key lies in how exponentiation treats deviations from the mean:
-
Odd powers (3rd moment for skewness): Preserve the sign of deviations. Negative deviations remain negative and positive ones remain positive. In many right-skewed distributions, large positive deviations dominate the cubed sum; the reverse often occurs for left skew. The probabilities and magnitudes on both sides still determine the sign.
-
Even powers (4th moment for kurtosis): Make all deviations nonnegative, while amplifying large deviations. Raising a deviation to the fourth power makes extreme observations contribute disproportionately to the sum.
The standardization by powers of makes these measures unitless. Moments provide compact shape summaries, but each summary discards detail, so plots and application-specific tail measures remain important.