Simple Linear Regression › Example

Let's work through a concrete example with actual numbers to see how simple linear regression works step by step. Suppose you're studying the relationship between study hours and test scores. You have collected the following data from five students:

Step 1: Calculate the meansLink Copied

First, we need to find the average values of both variables:

$$\bar{x} = \frac{1 + 2 + 3 + 4 + 5}{5} = \frac{15}{5} = 3$$ $$\bar{y} = \frac{65 + 70 + 80 + 85 + 90}{5} = \frac{390}{5} = 78$$

Step 2: Calculate the slope ($\beta_1$)Link Copied

Now we'll use the slope formula. Let's break this down into manageable pieces:

First, calculate $(x_i - \bar{x})(y_i - \bar{y})$ for each data point:

• Point 1: $(1 - 3)(65 - 78) = (-2)(-13) = 26$
• Point 2: $(2 - 3)(70 - 78) = (-1)(-8) = 8$
• Point 3: $(3 - 3)(80 - 78) = (0)(2) = 0$
• Point 4: $(4 - 3)(85 - 78) = (1)(7) = 7$
• Point 5: $(5 - 3)(90 - 78) = (2)(12) = 24$

Numerator: $26 + 8 + 0 + 7 + 24 = 65$

Next, calculate $(x_i - \bar{x})^2$ for each data point:

• Point 1: $(1 - 3)^2 = (-2)^2 = 4$
• Point 2: $(2 - 3)^2 = (-1)^2 = 1$
• Point 3: $(3 - 3)^2 = (0)^2 = 0$
• Point 4: $(4 - 3)^2 = (1)^2 = 1$
• Point 5: $(5 - 3)^2 = (2)^2 = 4$

Denominator: $4 + 1 + 0 + 1 + 4 = 10$

Therefore:

$$\beta_1 = \frac{65}{10} = 6.5$$

Step 3: Calculate the intercept ($\beta_0$)Link Copied

Using the intercept formula:

$$\beta_0 = \bar{y} - \beta_1 \bar{x} = 78 - 6.5 \times 3 = 78 - 19.5 = 58.5$$

Our final fitted equation is:

$$y = 58.5 + 6.5x$$

Step 4: Make a predictionLink Copied

If a student studies for 6 hours, their predicted test score would be:

$$y = 58.5 + 6.5 \times 6 = 58.5 + 39 = 97.5$$

Step 5: Interpret the resultsLink Copied

Our results can be interpreted as follows.

• Intercept (58.5): A student who studies 0 hours would be expected to score 58.5 points (though this might not be realistic in practice)
• Slope (6.5): For each additional hour of study, the test score increases by an average of 6.5 points
• Prediction: A student studying 6 hours would be expected to score about 97.5 points

The reason that we can simply interpret the slope and intercept as the change in the test score for each additional hour of study and the baseline score for no study, respectively, is because the linear model is a linear function of the form $y = \beta_0 + \beta_1 x$. This is why linear regression models are so interpretable.

Let's visualize this example to see how our calculated regression line fits the data and how we can use it to make predictions. The plot will show the original data points, the fitted regression line, and a prediction for a student studying 6 hours. This visualization helps confirm that our manual calculations are correct and demonstrates the practical application of simple linear regression.