P-values and Hypothesis Test Setup › Summary
Machine Learning from ScratchAll chapters P-values and Hypothesis Test Setup › SummaryOne section · Sections in this chapter7 / 9 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
This chapter established the foundations of hypothesis testing:
-
The logic: We assume the null hypothesis is true and ask how surprising our data would be under that assumption. If the data are surprising enough (low p-value), we reject the null.
-
P-values: The probability of observing data as extreme as ours, assuming the null is true. P-values measure evidence against the null, not the probability the null is true, and not the size of any effect.
-
Hypotheses: The null () is the skeptical position; the alternative () is what we're trying to establish. We never accept ; we either reject it or fail to reject it.
-
One-sided vs two-sided: Two-sided tests are the default. Use one-sided only when effects in one direction are meaningless.
-
Test statistics: Standardize the observed deviation by the expected variability. This allows comparison across different contexts and scales.
-
Decisions: Compare the p-value to your significance level α. Report exact p-values, not just "significant" or "not significant."