Multicollinearity › Multicollinearity: Understanding Variable Relationships
Machine Learning from ScratchAll chapters Multicollinearity › Multicollinearity: Understanding Variable RelationshipsOne section · Sections in this chapter1 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
When we build regression models with multiple predictor variables, we sometimes encounter a subtle but important problem: our predictor variables start talking to each other. This phenomenon, known as multicollinearity, occurs when two or more predictor variables in a regression model are highly correlated with each other, creating a web of relationships that makes it difficult to assess the individual effect of each variable.
Think of it like trying to understand the individual contributions of team members when they work so closely together that their efforts become indistinguishable. In regression analysis, predictor variables are the input variables we use to predict an outcome, and correlation describes how closely two variables move together, with values ranging from -1 to +1. When these predictors become too intertwined, our regression model.the statistical method designed to uncover relationships between variables and outcomes.struggles to separate their individual contributions.