Generalized Linear Models › Common GLM Types

When the response variable is continuous and normally distributed, and the relationship between the predictors and the response is assumed to be linear, we use the identity link function. This is the classic linear regression model.

• Distribution: Normal (Gaussian)
• Link function: $g(\mu) = \mu$ (identity)
• Model: $\mu = \beta_0 + \beta_1 x_1 + \cdots + \beta_p x_p$

where:

• $\mu = E[Y|X]$ is the expected value of the response variable $Y$ given the predictors $X$ (the conditional mean we're modeling)
• $\beta_0$ is the intercept (the expected response when all predictors equal zero)
• $\beta_1, \ldots, \beta_p$ are the regression coefficients (quantify the linear effect of each predictor)
• $x_1, \ldots, x_p$ are the predictor variables (the feature values)
• $p$ is the number of predictors

The identity link means that the expected response is modeled directly as a linear function of the features. This is the standard linear regression we've covered in previous sections, and it is the basis for all GLMs.

When to use: Use linear regression when your outcome is continuous with no fixed bounds and follows an approximately normal distribution. Examples include predicting house prices, heights, or temperatures.

When the response variable is binary (e.g., success/failure, yes/no, 0/1), logistic regression is the appropriate GLM. It uses the logit link function to model the probability of success.

• Distribution: Binomial
• Link function: $g(\mu) = \log\left(\frac{\mu}{1-\mu}\right)$ (logit)
• Model: $\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 x_1 + \cdots + \beta_p x_p$

where:

• $p = P(Y=1|X)$ is the probability that the response variable equals 1 (success) given predictors $X$ (the parameter we're modeling)
• $\frac{p}{1-p}$ is the odds of success (the ratio of the probability of success to the probability of failure)
• $\log\left(\frac{p}{1-p}\right)$ is the log-odds or logit (the natural logarithm of the odds)
• $\beta_0$ is the intercept (the log-odds when all predictors equal zero)
• $\beta_1, \ldots, \beta_p$ are the regression coefficients (represent the change in log-odds for a one-unit increase in each predictor)
• $x_1, \ldots, x_p$ are the predictor variables (the feature values)

The logit link transforms probabilities (which are constrained to be between 0 and 1) to the entire real line, allowing us to model them with a linear predictor.

When to use: Use logistic regression for binary classification tasks, such as predicting whether a customer will buy a product (yes/no), or whether an email is spam (spam/not spam).

For modeling count data.where the response variable represents the number of times an event occurs.Poisson regression is used. It employs the log link function to ensure that the predicted counts are non-negative.

• Distribution: Poisson
• Link function: $g(\mu) = \log(\mu)$ (log)
• Model: $\log(\lambda) = \beta_0 + \beta_1 x_1 + \cdots + \beta_p x_p$

where:

• $\lambda = E[Y|X]$ is the expected count (mean of the Poisson distribution) given predictors $X$ (the rate parameter we're modeling)
• $\log(\lambda)$ is the natural logarithm of the expected count (the link function applied to the mean)
• $\beta_0$ is the intercept (the log of the expected count when all predictors equal zero)
• $\beta_1, \ldots, \beta_p$ are the regression coefficients (represent the change in log-count for a one-unit increase in each predictor)
• $x_1, \ldots, x_p$ are the predictor variables (the feature values)

The log link ensures that the predicted mean is positive. By inverting the log link, we get:

$$\lambda = e^{\beta_0 + \beta_1 x_1 + \cdots + \beta_p x_p}$$

where:

• $e$ is Euler's number (approximately 2.718)
• The exponential function $e^x$ is positive for any real value $x$
• This ensures $\lambda > 0$, which is necessary for count data (counts cannot be negative)

When to use: Use Poisson regression when your outcome is a count (e.g., number of customer visits, number of accidents, number of emails received in an hour), and the counts are non-negative integers.

Linear and logistic regression, together with Poisson regression, are the most common and foundational examples of GLMs. Each is defined by its choice of distribution and link function, allowing you to tailor the model to the nature of your response variable.