Generalized Linear Models › Formula

The general GLM formula can be written as:

$$g(E[Y|X]) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_p x_p$$

where:

• $E[Y|X]$ is the expected value of the response variable $Y$ given the features $X$ (the conditional mean)
• $g(\cdot)$ is the link function (transforms the expected response to the linear predictor scale)
• $\beta_0$ is the intercept term
• $\beta_1, \ldots, \beta_p$ are the regression coefficients (one for each feature)
• $x_1, x_2, \ldots, x_p$ are the feature values (predictor variables)
• $p$ is the number of features

This reads as: "The link function applied to the expected value of the response equals the linear combination of features with their coefficients."

In matrix form, the GLM can be written as:

$$g(\boldsymbol{\mu}) = \mathbf{X}\boldsymbol{\beta}$$

where:

• $\boldsymbol{\mu} = E[\mathbf{Y}]$ is the vector of expected responses (one element per observation, where $\boldsymbol{\mu} = [\mu_1, \mu_2, \ldots, \mu_n]^T$ and $\mu_i = E[Y_i]$)
• $\mathbf{X}$ is the $n \times (p+1)$ design matrix of features (each row is one observation, each column is one feature, with the first column typically being all 1s for the intercept)
• $\boldsymbol{\beta}$ is the $(p+1) \times 1$ vector of coefficients ($\boldsymbol{\beta} = [\beta_0, \beta_1, \ldots, \beta_p]^T$)
• $g(\cdot)$ is applied element-wise to $\boldsymbol{\mu}$ (producing a vector where each element is $g(\mu_i)$)
• $n$ is the number of observations

The following diagram illustrates how the three components of a GLM connect to transform input features into predictions: