Weight Quantization Basics › Key Parameters
Michael BrenndoerferJanuary 10, 20261 min read
Language AI HandbookAll chapters Part 42 of 67Inference Optimization
Weight Quantization Basics › Key ParametersOne section · Sections in this chapter9 / 12 ↓
Sections in this chapter12
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for the quantization functions are:
- num_bits: The bit width for the target integer representation (e.g., 8 for INT8, 4 for INT4). Lower bits increase compression but reduce precision. The 4x-per-bit rule predicts the error cost of reducing bit width.
- axis: The dimension along which quantization parameters (scale, zero-point) are computed. For per-channel quantization of linear layers, this is typically the output dimension (axis 0), letting each output neuron to have its own scale.
- method: The strategy for determining quantization ranges during calibration. The choice between minmax and percentile-based calibration significantly affects accuracy when activation distributions have heavy tails.