GPTQ › Measuring Output Reconstruction Error
Language AI HandbookAll chapters GPTQ › Measuring Output Reconstruction ErrorOne section · Sections in this chapter11 / 17 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Layer output error is a better measure of quantization quality than weight error. Weight error measures how much the weights changed; output error measures how much the layer's predictions changed. Since the model's downstream behavior depends entirely on layer outputs rather than raw weights, output error is the relevant metric.
We want to verify that GPTQ's weight-space optimization translates into output-space improvements. If it did not, the Hessian-weighted objective would be clever mathematics without practical payoff. In practice, the improvement in output error is even more pronounced than the improvement in weight error, precisely because the Hessian weighting ensures we concentrate accuracy where the outputs are most sensitive.
def measure_output_error(original_weights, quantized_weights, inputs):
"""Compute the output reconstruction error ||W @ X - W_hat @ X||."""
original_out = inputs @ original_weights.T
quantized_out = inputs @ quantized_weights.T
mse = ((original_out - quantized_out) ** 2).mean().item()
return mse# Quantize all rows of the sample weight matrix
quantized_weights = torch.zeros_like(sample_weights)
for i in range(d_out):
q_row, params = gptq_quantize_row(sample_weights[i], H_inv)
# Dequantize for error measurement
for col in range(d_in):
group_idx = col // 128
scale, zero_point = params[group_idx]
quantized_weights[i, col] = dequantize_weight(
q_row[col].item(), scale, zero_point
)# Measure output reconstruction error
gptq_output_error = measure_output_error(
sample_weights, quantized_weights, sample_inputs
)
# Compare with naive quantization
naive_weights = torch.zeros_like(sample_weights)
for i in range(d_out):
row = sample_weights[i]
scale = (row.max() - row.min()) / 15
zp = round(-row.min().item() / scale.item()) if scale > 0 else 0
q = torch.round(row / scale + zp).clamp(0, 15)
naive_weights[i] = scale * (q - zp)
naive_output_error = measure_output_error(
sample_weights, naive_weights, sample_inputs
)
error_ratio = naive_output_error / gptq_output_errorGPTQ output reconstruction MSE: 0.022085 Naive output reconstruction MSE: 0.030633 GPTQ reaches 1.39x lower output error
The improvement in output reconstruction error is even more pronounced than the improvement in weight error, which is exactly what the Hessian-weighted loss function is designed to produce.
