Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

The Z-Test › Z-Test for Proportions

Michael BrenndoerferUpdated January 12, 20265 min read

Machine Learning from ScratchAll chapters

Part 2 of 9Hypothesis Testing

Chapter 14The Z-Test › Z-Test for Proportions

Reading section 7 of 11 · Z-Test for Proportions · 5 minSection 7 of 11 · 5 min

Section 7 of 11 · 5 min read

The Z-Test › Z-Test for ProportionsOne section · Sections in this chapter7 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The z-test for proportions is the most common real-world application of z-tests. It's used extensively in A/B testing, survey analysis, clinical trials, and quality control.

Why Z-Tests Work for ProportionsLink Copied

For a binomial proportion, the variance is determined by the proportion itself:

\text{Var}(\hat{p}) = \frac{p(1-p)}{n}

Under the null hypothesis $H_0: p = p_0$ , we use $p_0$ to calculate the standard error:

SE = \sqrt{\frac{p_0(1-p_0)}{n}}

The test statistic is:

z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}

where $\hat{p} = x/n$ is the sample proportion (x successes out of n trials).

When Is the Normal Approximation Valid?Link Copied

The z-test for proportions uses the normal approximation to the binomial. This works well when:

$n \cdot p_0 \geq 10$ (expected successes)
$n \cdot (1 - p_0) \geq 10$ (expected failures)

For smaller samples or proportions near 0 or 1, use exact binomial tests instead.

Example: A/B TestingLink Copied

An e-commerce company wants to test whether a new checkout design improves their conversion rate. Their current conversion rate is 3.2%. After showing the new design to 5000 visitors, 185 converted. Is there evidence of improvement?

In[10]:

Code

import numpy as np
from scipy import stats

# Data
successes = 185
n = 5000
p_0 = 0.032  # Current/null conversion rate

# Sample proportion
p_hat = successes / n

# Standard error under null hypothesis
se = np.sqrt(p_0 * (1 - p_0) / n)

# Z-statistic
z = (p_hat - p_0) / se

# One-sided p-value (testing for improvement)
p_value = stats.norm.sf(z)

# Check normal approximation conditions
expected_successes = n * p_0
expected_failures = n * (1 - p_0)

print("Proportion Z-Test: A/B Test")
print("=" * 50)
print(f"Sample: {successes} conversions out of {n} visitors")
print(f"Sample proportion: {p_hat:.4f} ({p_hat * 100:.2f}%)")
print(f"Null hypothesis: p = {p_0} ({p_0 * 100:.1f}%)")
print()
print(f"Standard error: {se:.5f}")
print(f"Z-statistic: {z:.3f}")
print(f"P-value (one-sided, right): {p_value:.4f}")
print()
print("Normal approximation check:")
print(
    f"  Expected successes: {expected_successes:.1f} ≥ 10? {'✓' if expected_successes >= 10 else '✗'}"
)
print(
    f"  Expected failures: {expected_failures:.1f} ≥ 10? {'✓' if expected_failures >= 10 else '✗'}"
)
print()
print(f"Decision: {'Reject H₀' if p_value < 0.05 else 'Fail to reject H₀'}")

Out[10]:

Console

Proportion Z-Test: A/B Test
==================================================
Sample: 185 conversions out of 5000 visitors
Sample proportion: 0.0370 (3.70%)
Null hypothesis: p = 0.032 (3.2%)

Standard error: 0.00249
Z-statistic: 2.009
P-value (one-sided, right): 0.0223

Normal approximation check:
  Expected successes: 160.0 ≥ 10? ✓
  Expected failures: 4840.0 ≥ 10? ✓

Decision: Reject H₀

The p-value of about 0.022 is less than 0.05, so we have evidence that the new design improves conversion rates. The improvement from 3.2% to 3.7% represents a 15.6% relative increase.

Out[11]:

Visualization

Normal distribution showing sampling distribution for proportion with observed value and rejection region marked. — Visualization of the A/B test result. The observed conversion rate of 3.7% corresponds to a z-statistic of 2.01 and a one-sided p-value of 0.0223, which falls in the rejection region at α = 0.05.

Two-Sample Proportion TestLink Copied

When comparing proportions from two groups (e.g., control vs treatment), use:

z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}

Where $\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$ is the pooled proportion under the null hypothesis that $p_1 = p_2$ .

In[12]:

Code

import numpy as np
from scipy import stats

# Example: Comparing two treatments
# Treatment A: 45 successes out of 200
# Treatment B: 65 successes out of 200
x1, n1 = 45, 200
x2, n2 = 65, 200

# Sample proportions
p1 = x1 / n1
p2 = x2 / n2

# Pooled proportion
p_pooled = (x1 + x2) / (n1 + n2)

# Standard error
se = np.sqrt(p_pooled * (1 - p_pooled) * (1 / n1 + 1 / n2))

# Z-statistic
z = (p1 - p2) / se

# Two-sided p-value
p_value = 2 * stats.norm.sf(abs(z))

print("Two-Proportion Z-Test: Treatment Comparison")
print("=" * 50)
print(f"Treatment A: {x1}/{n1} = {p1:.3f} ({p1 * 100:.1f}%)")
print(f"Treatment B: {x2}/{n2} = {p2:.3f} ({p2 * 100:.1f}%)")
print()
print(f"Difference: {(p2 - p1) * 100:.1f} percentage points")
print(f"Pooled proportion: {p_pooled:.3f}")
print(f"Standard error: {se:.4f}")
print(f"Z-statistic: {z:.3f}")
print(f"P-value (two-sided): {p_value:.4f}")
print()
print(f"Decision: {'Reject H₀' if p_value < 0.05 else 'Fail to reject H₀'}")

Out[12]:

Console

Two-Proportion Z-Test: Treatment Comparison
==================================================
Treatment A: 45/200 = 0.225 (22.5%)
Treatment B: 65/200 = 0.325 (32.5%)

Difference: 10.0 percentage points
Pooled proportion: 0.275
Standard error: 0.0447
Z-statistic: -2.240
P-value (two-sided): 0.0251

Decision: Reject H₀