Descriptive Statistics › Measures of Central Tendency
Machine Learning from ScratchAll chapters Descriptive Statistics › Measures of Central TendencyOne section · Sections in this chapter3 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Central tendency measures answer a key question: what is a typical value in this dataset? The mean and median answer this question for numerical data, while the mode identifies the most frequent value. The right choice depends on the data's properties and the analysis goals [2].
The Mean
The arithmetic mean, commonly called the average, is calculated by summing all values and dividing by the count of observations:
Where:
- : Sample mean
- : Individual observation
- : Number of observations
The mean represents the balance point of a distribution and incorporates information from every data point. Under a normal model it is an efficient estimator of location, but symmetry alone is not enough to guarantee that property [2]. Its sensitivity to all values also makes it vulnerable to outliers: a single extremely large or small value can pull the mean far from where most observations cluster.
For example, consider household income in a neighborhood where most families earn between 80,000 annually, but one household earns 150,000, far above what most residents earn, while the median would remain around $60,000, better representing the typical household.
The Median
The median is the middle value when observations are arranged in order. For datasets with an odd number of observations, it is simply the center value; for even counts, it is the average of the two middle values:
Where denotes the -th value in the sorted dataset.
For example, consider the sorted dataset: 12, 15, 18, 22, 25, 28, 31. With seven observations (odd count), the median is the fourth value: 22. If we add an eighth observation to get 12, 15, 18, 22, 25, 28, 31, 45, the median becomes the average of the fourth and fifth values: (22 + 25) / 2 = 23.5.
The median's main advantage is its resistance to the magnitude of extreme tail values. Moving an already extreme observation farther into the tail does not change its rank and usually leaves the median unchanged; changing observations near the middle can still change it. This makes the median particularly useful for skewed distributions or data with outliers.
The Mode
For a discrete sample, the mode is any value with the highest observed frequency. Two values tied for that maximum give two sample modes, as when exam scores of 75 and 90 each occur eight times and no other score occurs more than five times. For a continuous distribution, modes are peaks or local maxima of its density. Multiple peaks may suggest a mixture of subgroups, but they do not prove one, and the peaks seen in a histogram or KDE can change with bin width or smoothing bandwidth [3]. If every observed value has the same frequency, all values formally tie; many introductory summaries report that situation as having no unique mode.
The mode is especially useful for nominal categorical data, where mean and median are undefined, such as the most common color preference in a survey or the most frequent diagnosis in a medical dataset. Ordered categorical data may also support a median. For continuous numerical data, estimated density modes can help identify peaks, although their locations depend on the estimation method.
In practice, for continuous data, we often examine the mode through histograms or kernel density estimates rather than computing the exact most frequent value, especially when data have many unique values.
Kernel density estimation (KDE) is a nonparametric method for estimating a probability density. It centers a kernel, often Gaussian, on each observation and averages the bandwidth-scaled kernels to form a smooth curve. KDE is useful for exploring continuous distributions and possible modes when exact repetitions are rare, but the selected bandwidth materially affects how many peaks appear [4].
Don't worry if this concept seems unclear at this point. It's not essential to understand right now. Feel free to continue and revisit it later if needed.