Poisson Regression › Example

Let's work through a concrete example to understand how Poisson regression operates step by step. Suppose we're modeling the number of customer complaints received per day at a retail store, and we have two predictors: the number of customers served (in hundreds) and whether it's a weekend (1 for weekend, 0 for weekday).

Our dataset contains the following observations:

The Poisson regression model assumes that the logarithm of the expected number of complaints is linearly related to our predictors:

$$\log(\lambda_i) = \beta_0 + \beta_1 \cdot \text{Customers}_i + \beta_2 \cdot \text{Weekend}_i$$

where:

• $\lambda_i$ is the expected number of complaints for day $i$ (the mean of the Poisson distribution for observation $i$; constrained to be positive)
• $\beta_0$ is the intercept (the log of the expected count when all predictors equal zero; baseline log-count)
• $\beta_1$ is the coefficient for customer count (the change in log-count for each additional hundred customers, holding weekend status constant)
• $\beta_2$ is the coefficient for weekend indicator (the change in log-count when comparing weekends to weekdays, holding customer count constant)
• $\text{Customers}_i$ is the number of customers (in hundreds) for day $i$ (a continuous predictor variable)
• $\text{Weekend}_i$ is 1 if day $i$ is a weekend, 0 otherwise (a binary indicator variable)

This means the expected count is:

$$\lambda_i = \exp(\beta_0 + \beta_1 \cdot \text{Customers}_i + \beta_2 \cdot \text{Weekend}_i)$$

where:

• $\exp(\cdot)$ is the exponential function (ensures $\lambda_i > 0$ for any values of the coefficients and predictors)
• This is the inverse transformation of the log-link, converting from the log scale back to the count scale

Let's say our fitted model produces the following coefficients: $\beta_0 = 0.2$, $\beta_1 = 0.8$, and $\beta_2 = -0.5$. Now we can calculate the expected number of complaints for each day.

• For Day 1:

  $$\lambda_1 = \exp(0.2 + 0.8 \times 2.5 + (-0.5) \times 0) = \exp(0.2 + 2.0 + 0) = \exp(2.2) \approx 9.03$$

  where: $\text{Customers}_1 = 2.5$, $\text{Weekend}_1 = 0$ (weekday), so the linear predictor is $0.2 + 0.8(2.5) + (-0.5)(0) = 2.2$

• For Day 2:

  $$\lambda_2 = \exp(0.2 + 0.8 \times 3.0 + (-0.5) \times 0) = \exp(0.2 + 2.4 + 0) = \exp(2.6) \approx 13.46$$

  where: $\text{Customers}_2 = 3.0$, $\text{Weekend}_2 = 0$ (weekday), so the linear predictor is $0.2 + 0.8(3.0) + (-0.5)(0) = 2.6$

• For Day 3:

  $$\lambda_3 = \exp(0.2 + 0.8 \times 1.8 + (-0.5) \times 1) = \exp(0.2 + 1.44 - 0.5) = \exp(1.14) \approx 3.13$$

  where: $\text{Customers}_3 = 1.8$, $\text{Weekend}_3 = 1$ (weekend), so the linear predictor is $0.2 + 0.8(1.8) + (-0.5)(1) = 1.14$

• For Day 4:

  $$\lambda_4 = \exp(0.2 + 0.8 \times 4.2 + (-0.5) \times 0) = \exp(0.2 + 3.36 + 0) = \exp(3.56) \approx 35.16$$

  where: $\text{Customers}_4 = 4.2$, $\text{Weekend}_4 = 0$ (weekday), so the linear predictor is $0.2 + 0.8(4.2) + (-0.5)(0) = 3.56$

• For Day 5:

  $$\lambda_5 = \exp(0.2 + 0.8 \times 2.1 + (-0.5) \times 1) = \exp(0.2 + 1.68 - 0.5) = \exp(1.38) \approx 3.97$$

  where: $\text{Customers}_5 = 2.1$, $\text{Weekend}_5 = 1$ (weekend), so the linear predictor is $0.2 + 0.8(2.1) + (-0.5)(1) = 1.38$

Now let's interpret these coefficients. The coefficient $\beta_1 = 0.8$ means that for every additional hundred customers, the expected number of complaints increases by a factor of $\exp(0.8) \approx 2.23$. In other words, each additional hundred customers more than doubles the expected complaints.

Calculation:

$$e^{0.8} = 2.2255 \approx 2.23$$

This multiplicative effect means that if we have 200 customers instead of 100, we expect $2.23 \times$ as many complaints, holding weekend status constant.

The coefficient $\beta_2 = -0.5$ means that weekends have $\exp(-0.5) \approx 0.61$ times as many complaints as weekdays, holding customer count constant. This suggests that weekends have about 39% fewer complaints than weekdays, possibly because weekend customers are more relaxed or have different expectations.

Calculation:

$$e^{-0.5} = 0.6065 \approx 0.61$$ $$\text{Percent decrease} = (1 - 0.6065) \times 100\% = 39.35\% \approx 39\%$$

The intercept $\beta_0 = 0.2$ represents the log of the expected complaints when both predictors are zero (no customers on a weekday), which gives us $\exp(0.2) \approx 1.22$ expected complaints.a baseline level that makes sense for a retail environment.

Calculation:

$$e^{0.2} = 1.2214 \approx 1.22$$

In the example above, we simply stated the coefficients as $\beta_0 = 0.2$, $\beta_1 = 0.8$, and $\beta_2 = -0.5$ for illustration. But how do we determine these values from data?

As previously discussed, to estimate the coefficients in Poisson regression, we use maximum likelihood estimation (MLE). The idea is to find the values of $\beta_0$, $\beta_1$, and $\beta_2$ that make the observed data most probable under the Poisson model. This involves the following steps:

1. Specify the likelihood function: For each observation $i$, the probability of observing $y_i$ complaints given predictors $x_i$ is:

   $$P(Y_i = y_i \mid x_i) = \frac{\lambda_i^{y_i} e^{-\lambda_i}}{y_i!}$$

   where:

   • $Y_i$ is the random variable representing the complaint count for observation $i$ (a Poisson-distributed random variable)
   • $y_i$ is the observed complaint count for observation $i$ (the actual data value we observed; a non-negative integer)
   • $x_i$ represents the predictor values for observation $i$ (in our example, customer count and weekend indicator)
   • $\lambda_i = \exp(\beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2})$ is the expected count given the predictors (the mean parameter of the Poisson distribution for observation $i$)
   • $e$ is Euler's number (approximately 2.71828)

2. Write the log-likelihood: The log-likelihood for all $n$ observations is:

   $$\ell(\boldsymbol{\beta}) = \sum_{i=1}^n \left[ y_i \log(\lambda_i) - \lambda_i - \log(y_i!) \right]$$

   where:

   • $\ell(\boldsymbol{\beta})$ is the log-likelihood function (the natural logarithm of the likelihood; easier to maximize than the likelihood itself)
   • $\boldsymbol{\beta} = [\beta_0, \beta_1, \beta_2]^T$ is the vector of coefficients we want to estimate (the parameters of our model)
   • $n$ is the total number of observations (5 days in our example)
   • $\sum_{i=1}^n$ denotes summing over all observations (converting the product of probabilities to a sum of log-probabilities)
   • This function measures how well a given set of coefficients explains the observed data (higher values indicate better fit)

3. Maximize the log-likelihood: We seek the values of $\beta_0$, $\beta_1$, and $\beta_2$ that maximize $\ell(\boldsymbol{\beta})$. Mathematically, we solve:

   $$\hat{\boldsymbol{\beta}} = \arg\max_{\boldsymbol{\beta}} \ell(\boldsymbol{\beta})$$

   where:

   • $\hat{\boldsymbol{\beta}}$ denotes the maximum likelihood estimates (the optimal coefficient values)
   • $\arg\max$ means "the argument that maximizes" (the value of $\boldsymbol{\beta}$ that gives the highest log-likelihood)

   There is no closed-form solution, so we use numerical optimization algorithms (such as Newton-Raphson or iteratively reweighted least squares) to find the best-fitting coefficients.

4. Interpret the fitted coefficients: Once the optimization converges, the resulting coefficients are the maximum likelihood estimates $\hat{\beta}_0$, $\hat{\beta}_1$, and $\hat{\beta}_2$. These are the values that would be reported by statistical software or machine learning libraries when fitting a Poisson regression model to the data.

Note: In practice, you do not need to perform these calculations by hand. Statistical software (such as statsmodels or scikit-learn, or specialized packages) will fit the model and return the estimated coefficients, along with measures of uncertainty and model fit.

In summary, the coefficients in Poisson regression are determined by finding the values that maximize the likelihood of observing the data, given the model. This process ensures that the fitted model best captures the relationship between the predictors and the count outcome according to the Poisson distribution.