P-values and Hypothesis Test Setup › Setting Up a Hypothesis Test

With p-values understood, let's examine the mechanics of conducting a test. Every hypothesis test has the same structure: formulate hypotheses, choose a test, collect data, compute a test statistic, and make a decision.

The null hypothesis ($H_0$) represents the skeptical position. It typically claims "nothing interesting is happening": no effect, no difference, no relationship. The null is what we assume true unless the data provide strong evidence otherwise.

The alternative hypothesis ($H_1$ or $H_a$) is what we're trying to establish. It contradicts the null and represents the research claim.

Example 1: Drug efficacy

• $H_0$: The drug has no effect on blood pressure (mean change = 0)
• $H_1$: The drug affects blood pressure (mean change ≠ 0)

Example 2: A/B test

• $H_0$: The new design has the same conversion rate as the old (rate_new = rate_old)
• $H_1$: The new design has a different conversion rate (rate_new ≠ rate_old)

Example 3: Quality control

• $H_0$: The machine is calibrated correctly (mean weight = 50g)
• $H_1$: The machine is miscalibrated (mean weight ≠ 50g)

The alternative hypothesis can be directional (one-sided) or non-directional (two-sided).

A two-sided test considers alternatives in both directions:

• $H_0: \mu = \mu_0$
• $H_1: \mu \neq \mu_0$

Use this when deviations in either direction are scientifically meaningful. If you're testing whether a new drug affects blood pressure, you care whether it raises OR lowers it.

A one-sided test considers alternatives in only one direction:

• $H_0: \mu = \mu_0$ (or $\mu \leq \mu_0$)
• $H_1: \mu > \mu_0$ (right-tailed)

Or:

• $H_0: \mu = \mu_0$ (or $\mu \geq \mu_0$)
• $H_1: \mu < \mu_0$ (left-tailed)

Use this when only one direction matters. If you're testing whether a new treatment improves outcomes, you might not care if it makes things worse (that's a different problem).