Simple Linear Regression › Implementation in Scikit-learn
Machine Learning from ScratchAll chapters Simple Linear Regression › Implementation in Scikit-learnOne section · Sections in this chapter7 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Scikit-learn provides a clean and efficient implementation of simple linear regression that handles all the mathematical calculations automatically. We'll walk through a step-by-step implementation that verifies our manual calculations and demonstrates how to use this method in practice.
Step 1: Import Required Libraries
First, we need to import the necessary libraries for our implementation:
Step 2: Prepare the Data
Scikit-learn expects the feature matrix X to be 2D, even for a single feature. We'll prepare our study hours and test scores data:
# Prepare the data (scikit-learn expects 2D arrays)
X = np.array([1, 2, 3, 4, 5]).reshape(-1, 1) # Study hours
y = np.array([65, 70, 80, 85, 90]) # Test scoresLet's verify the data shapes to ensure proper formatting:
Feature matrix X shape: (5, 1) Target vector y shape: (5,)
The output shows that X has shape (5, 1), meaning 5 samples with 1 feature each. This is the correct format for scikit-learn. The y vector has shape (5,), representing 5 target values. This data preparation is important because scikit-learn's LinearRegression expects this specific input format.
Step 3: Create and Train the Model
Now we'll create a LinearRegression model and fit it to our data:
# Create and fit the model
model = LinearRegression()
model.fit(X, y)The model has been successfully trained. The fit() method automatically calculated the optimal coefficients using the least squares method we discussed earlier. This is typically more efficient than our manual calculations and handles edge cases automatically.
Step 4: Extract Model Parameters
Let's examine the fitted model parameters:
# Extract model parameters
intercept = model.intercept_
slope = model.coef_[0]Now let's display the fitted parameters:
Intercept (β₀): 58.5 Slope (β₁): 6.5 Fitted equation: y = 58.5 + 6.5x
These results match our manual calculations. The intercept and slope values are identical to what we calculated step-by-step. This confirms that our mathematical understanding is correct and that scikit-learn is implementing the same least squares method.
Step 5: Make Predictions
Now let's use the fitted model to make predictions:
# Make predictions for new data points
new_hours = [[6], [7], [8]] # Multiple predictions
predictions = model.predict(new_hours)Let's display the predictions:
Predictions for 6, 7, and 8 hours of study: 6 hours: 97.5 points 7 hours: 104.0 points 8 hours: 110.5 points
The predictions show a consistent increase per additional hour of study, which matches our slope coefficient. The prediction for 6 hours matches our manual calculation. These predictions appear reasonable and follow the linear pattern we established.
Step 6: Evaluate Model Performance
Let's assess how well our model fits the data:
# Calculate model performance metrics
r_squared = model.score(X, y)
# Calculate predictions for training data
y_pred = model.predict(X)
# Calculate mean squared error
mse = np.mean((y - y_pred) ** 2)
rmse = np.sqrt(mse)Now let's display the performance metrics:
R-squared: 0.983 Mean Squared Error: 1.50 Root Mean Squared Error: 1.22
The R-squared value indicates that the model explains most, but not all, of the variance in the training data. This represents strong performance - an R-squared above 0.9 is generally considered very good. The Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) values show that there is a small amount of error in the model's predictions. This is typical in real-world scenarios, where some error is expected due to noise and measurement uncertainty.
Key Parameters
Below are some of the main parameters that affect how the model works and performs.
fit_intercept: Whether to calculate the intercept for this model (default: True). Set to False only if you know the data is already centered or you want to force the line through the origin.normalize: Whether to normalize the regressors before regression (default: False). Deprecated in newer versions - use StandardScaler for preprocessing instead.copy_X: Whether to copy X or overwrite the original (default: True). Set to False to save memory when working with large datasets.
Key Methods
The following are the most commonly used methods for interacting with the model.
fit(X, y): Fits the linear model to training data. X should be 2D array-like with shape (n_samples, n_features), y should be 1D array-like with shape (n_samples,).predict(X): Predicts target values for new data points. Returns predictions as 1D array with shape (n_samples,).score(X, y): Returns the coefficient of determination R² of the prediction. Values closer to 1.0 indicate better fit.