Generalized Linear Models › Components of GLMs

A Generalized Linear Model consists of three main components:

The random component specifies the probability distribution of the response variable $Y$. GLMs assume that $Y$ follows a distribution from the exponential family, which includes:

• Normal (Gaussian): For continuous data
• Binomial: For binary data (success/failure)
• Poisson: For count data
• Gamma: For positive continuous data
• Inverse Gaussian: For positive continuous data with specific variance properties

The systematic component is the linear combination of features:

$$\eta = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_p x_p$$

where:

• $\eta$ is the linear predictor (a scalar value for a single observation, or a vector for multiple observations)
• $\beta_0$ is the intercept term (the expected response when all features equal zero)
• $\beta_1, \beta_2, \ldots, \beta_p$ are the regression coefficients for each feature (quantify the effect of each feature on the linear predictor)
• $x_1, x_2, \ldots, x_p$ are the feature values (predictor variables for a single observation)
• $p$ is the number of features (total count of predictor variables in the model)

This is identical to the linear predictor in ordinary linear regression.

The link function $g(\cdot)$ connects the linear predictor $\eta$ to the expected value of the response $\mu = E[Y]$:

$$g(\mu) = \eta$$

where:

• $g(\cdot)$ is the link function (a monotonic, differentiable transformation that maps the mean response to the linear predictor scale)
• $\mu = E[Y]$ is the expected value (mean) of the response variable $Y$ (the parameter we're modeling)
• $\eta$ is the linear predictor from the systematic component (the right-hand side of the equation: $\beta_0 + \beta_1 x_1 + \cdots + \beta_p x_p$)

The link function transforms the expected response so that it can be modeled linearly. Different GLM families use different link functions appropriate for their response distributions.