GPTQ › The Layer-Wise Reconstruction Objective

To understand how GPTQ approaches quantization, we must first establish what it means to quantize well. The basic question is: what objective should we optimize? One natural answer might be to minimize the difference between original and quantized weights directly. However, this ignores a important insight: not all weight errors are equally harmful. What truly matters is how quantization affects the layer's output, because the output is what downstream computations depend upon. A weight that multiplies a small input contributes little to the output regardless of its quantization error; a weight that multiplies a large, frequently-occurring input is necessary and must be handled with care.

GPTQ operates on one layer at a time, treating each layer's quantization as an independent optimization problem. This layer-wise decomposition is both a practical necessity and a reasonable approximation. It is practical because computing gradients through the entire model for every quantization decision would require a full backward pass per weight, which is completely infeasible for models with billions of parameters. It is a reasonable approximation because the output of each layer is the input to the next, and minimizing the error introduced at each layer independently tends to minimize the total error propagated through the network.

Think of it as optimizing each department of a large company independently. You cannot simultaneously optimize every decision across every team, but if each department minimizes its own errors, the overall company performance tends to be much better than if no one cared about local quality. The layer-wise approach gives GPTQ a tractable decomposition of an otherwise intractable global problem.

Consider a linear layer with weights $\mathbf{W} \in \mathbb{R}^{d_{out} \times d_{in}}$ and inputs $\mathbf{X} \in \mathbb{R}^{d_{in} \times n}$, where $n$ is the number of calibration tokens we use to estimate the layer's behavior. The goal is to find quantized weights $\hat{\mathbf{W}}$ that minimize the reconstruction error:

$$\mathcal{L}(\hat{\mathbf{W}}) = ||\mathbf{W}\mathbf{X} - \hat{\mathbf{W}}\mathbf{X}||_F^2$$

where:

• $\mathcal{L}(\hat{\mathbf{W}})$: the reconstruction error (loss) for the quantized weights
• $\mathbf{W}$: the original high-precision weight matrix
• $\hat{\mathbf{W}}$: the quantized weight matrix we are searching for
• $\mathbf{X}$: the input matrix containing calibration data (each column is one token's activations)
• $||\cdot||_F$: the Frobenius norm, equal to the square root of the sum of all squared elements

This loss function measures how much the layer's output changes due to quantization. We are comparing outputs, not weights directly. This distinction is needed: a weight that interacts with large input values will have a greater impact on the output than a weight that sees only small inputs. Two weights might differ by the same absolute amount from their quantized versions, but if one multiplies inputs that are ten times larger, it creates ten times more output distortion. The Frobenius norm aggregates all of these output differences into a single scalar that we can minimize.

The reconstruction error can be rewritten in a form that reveals important mathematical structure. This reformulation exposes the role of input correlations in determining which weights matter most. Expanding the Frobenius norm using the identity $||\mathbf{A}||_F^2 = \text{Tr}(\mathbf{A}\mathbf{A}^T)$:

$$\begin{aligned} \mathcal{L}(\hat{\mathbf{W}}) &= ||(\mathbf{W} - \hat{\mathbf{W}})\mathbf{X}||_F^2 \\ &= \text{Tr}\left[ ((\mathbf{W} - \hat{\mathbf{W}})\mathbf{X}) ((\mathbf{W} - \hat{\mathbf{W}})\mathbf{X})^T \right] && \text{(apply identity: } ||\mathbf{A}||_F^2 = \text{Tr}(\mathbf{A}\mathbf{A}^T)\text{)} \\ &= \text{Tr}\left[(\mathbf{W} - \hat{\mathbf{W}})\mathbf{X}\mathbf{X}^T(\mathbf{W} - \hat{\mathbf{W}})^T\right] && \text{(expand transpose: } (\mathbf{AB})^T = \mathbf{B}^T\mathbf{A}^T\text{)} \end{aligned}$$

where:

• $\text{Tr}[\cdot]$: the trace operator, which sums the diagonal elements of a square matrix
• $(\mathbf{W} - \hat{\mathbf{W}})$: the weight error matrix, capturing how much each weight was shifted by quantization
• $\mathbf{X}^T$: the transpose of the input matrix

The trace operation sums the diagonal elements of the resulting matrix. What emerges from this manipulation is that the loss depends on the weight errors not in isolation, but as weighted by the matrix $\mathbf{X}\mathbf{X}^T$. This matrix captures how the inputs correlate with each other: if two input dimensions tend to activate together, errors in their corresponding weights will interact. The matrix $\mathbf{X}\mathbf{X}^T$ is in effect the unnormalized sample covariance of the layer's inputs, computed across all calibration tokens.

Define $\mathbf{H} = 2\mathbf{X}\mathbf{X}^T$, which we call the Hessian matrix (we'll explain why this name is appropriate in the next section). The loss becomes:

$$\mathcal{L}(\hat{\mathbf{W}}) = \frac{1}{2}\text{Tr}\left[(\mathbf{W} - \hat{\mathbf{W}})\mathbf{H}(\mathbf{W} - \hat{\mathbf{W}})^T\right]$$

where:

• $\text{Tr}[\cdot]$: the trace operator
• $(\mathbf{W} - \hat{\mathbf{W}})$: the weight error matrix
• $\mathbf{H}$: the Hessian matrix ($2\mathbf{X}\mathbf{X}^T$) that captures input correlations
• $\frac{1}{2}$: a scaling factor arising from the definition of $\mathbf{H}$

This quadratic form indicates that the loss is a weighted sum of squared errors, where the weighting comes from the Hessian matrix. Errors in weights that correspond to highly active or correlated inputs are penalized more heavily than errors in weights that rarely contribute to the output. The quadratic structure is not merely convenient notation: it is what lets GPTQ to compute closed-form solutions for optimal corrections rather than relying on iterative gradient descent.

Since the rows of $\mathbf{W}$ don't interact in this expression (each output dimension is computed independently as a dot product with the inputs), we can optimize each row separately. This further simplifies our problem: instead of optimizing over all $d_{out} \times d_{in}$ weights simultaneously, we can solve $d_{out}$ independent problems, each involving only $d_{in}$ weights. For a single row $\mathbf{w} \in \mathbb{R}^{d_{in}}$:

$$\mathcal{L}(\hat{\mathbf{w}}) = \frac{1}{2}(\mathbf{w} - \hat{\mathbf{w}})\mathbf{H}(\mathbf{w} - \hat{\mathbf{w}})^T$$

where:

• $\mathbf{w}$: a single row of the original weight matrix (one output neuron's incoming weights)
• $\hat{\mathbf{w}}$: the corresponding row of quantized weights
• $\mathbf{H}$: the Hessian matrix, shared across all rows because all output neurons see the same inputs
• $(\mathbf{w} - \hat{\mathbf{w}})$: the error vector for this row

This weighted squared error gives us the foundation for GPTQ's optimization strategy. The quadratic structure of this loss function is important because it lets closed-form solutions for optimal weight updates. We do not need to run gradient descent or perform any iterative optimization; we can compute the exact optimal correction in a single step, which is what makes GPTQ both theoretically clean and computationally efficient.