Weight Quantization Basics › Symmetric vs Asymmetric Quantization

The two main quantization schemes differ in how they handle the zero-point. This choice significantly affects both accuracy and computational efficiency. Understanding when to use each scheme requires considering both the mathematical properties of the data and the practical constraints of the target hardware. The schemes represent different points on a tradeoff curve between representational flexibility and computational simplicity.

Symmetric quantization takes a simpler approach by forcing the zero-point to be zero ($z = 0$), which centers the quantized range around zero. This constraint means that real zero always maps to integer zero, creating a symmetric mapping where positive and negative values are treated identically. The scale is determined by the largest absolute value in the tensor:

$$s = \frac{\max(|x|)}{q_\text{max}}$$

where:

• $s$: the scale factor
• $\max(|x|)$: the maximum absolute value present in the tensor
• $q_\text{max}$: the maximum positive integer in the target range

This formula ensures that the largest magnitude value, whether positive or negative, maps to the edge of the integer range. The symmetry comes from the fact that $-\max(|x|)$ would map to $-q_\text{max}$, using the same scale factor. The entire representation is anchored at zero, which mirrors the typical shape of neural network weight distributions.

For 8-bit signed integers with $q_\text{max} = 127$, a tensor with values in $[-0.5, 0.3]$ would use $s = 0.5 / 127 \approx 0.00394$. This means:

• $-0.5$ maps to $-127$
• $0.0$ maps to $0$
• $0.3$ maps to $\text{round}(0.3 / 0.00394) = 76$

The main advantage of symmetric quantization is computational simplicity. With $z = 0$, the dequantization becomes a simple multiplication without any addition:

$$\hat{x} = s \cdot q$$

where:

• $\hat{x}$: the reconstructed real value
• $s$: the scale factor
• $q$: the quantized integer

This simplification significantly improves computational efficiency. Matrix multiplications can stay in integer arithmetic longer before converting back to floating point. When multiplying two symmetrically quantized matrices $A$ and $B$ with scales $s_A$ and $s_B$, the entire integer multiplication proceeds first, then the combined scale $s_A \cdot s_B$ is applied to the integer result. This allows modern hardware to use its specialized integer multiply-accumulate units for the bulk of the computation. Many hardware accelerators, including tensor cores on NVIDIA GPUs, are heavily optimized for this pattern.

However, symmetric quantization wastes representable range when the tensor distribution is asymmetric. In our example, values only range up to 0.3, but we allocated integer range up to 76, leaving integers 77 through 127 completely unused. Those 51 integer levels sit idle, representing precision budget that we paid for in bits but never used. The scale of 0.00394 is larger than it needs to be given the actual positive value range, meaning each step in the positive direction is coarser than necessary.

This wastefulness matters more at lower bit widths. With 8 bits, wasting 20% of the range costs little. With 4 bits, you have only 16 distinct positive values, and wasting a few is expensive. As we push toward 4-bit quantization, the precision budget becomes tight enough that asymmetric schemes often become worth their added complexity.

Asymmetric quantization removes the symmetry constraint, letting a non-zero $z$ and thereby using the full integer range for the actual value distribution. This flexibility comes at the cost of additional complexity but provides better precision when the data distribution is skewed.

The key insight is that real zero does not need to map to integer zero. It only needs to map to some integer, and we store that integer (the zero-point) so we can undo the shift during dequantization. By letting this shift, we can slide the integer range to perfectly bracket the actual range of values, with no wasted integers at either end.

For our tensor with values in $[-0.5, 0.3]$, using signed 8-bit integers ($q_\text{min} = -128$, $q_\text{max} = 127$):

$$\begin{aligned} s &= \frac{0.3 - (-0.5)}{127 - (-128)} && \text{(total real range / total integer steps)} \\ &= \frac{0.8}{255} \\ &\approx 0.00314 \end{aligned}$$ $$\begin{aligned} z &= -128 - \text{round}\left(\frac{-0.5}{0.00314}\right) && \text{(substituting into } z = q_{\text{min}} - \text{round}(x_{\text{min}}/s) \text{)} \\ &= -128 - (-159) \\ &= 31 \end{aligned}$$

Now the mapping looks like:

• $-0.5$ maps to $\text{round}(-0.5 / 0.00314) + 31 = -159 + 31 = -128$
• $0.0$ maps to $\text{round}(0 / 0.00314) + 31 = 31$
• $0.3$ maps to $\text{round}(0.3 / 0.00314) + 31 = 96 + 31 = 127$

The full integer range from $-128$ to $127$ is utilized. The scale is 0.00314 instead of 0.00394, meaning each integer step represents a smaller real interval and quantization error is reduced. This improvement of approximately 20% in scale translates directly to a 20% reduction in maximum quantization error per value. For a 4-bit system where every level counts, this 20% improvement can make the difference between acceptable and unacceptable accuracy loss.

The tradeoff is computational complexity. The zero-point must be tracked and subtracted during dequantization or matrix operations. For a simple dequantization, this adds just one subtraction. But for matrix multiplications, the situation is more complex. When multiplying two asymmetrically quantized matrices with values $q_A - z_A$ and $q_B - z_B$, the product expands into cross-terms involving the zero-points:

$$(q_A - z_A)(q_B - z_B) = q_A q_B - z_A q_B - q_A z_B + z_A z_B$$

The term $q_A q_B$ can be computed efficiently using integer hardware. The constant term $z_A z_B$ can be precomputed. But the terms $z_A q_B$ and $q_A z_B$ require multiplying integers by constants, which adds overhead. This overhead does not affect the weight quantization use case as severely, since weights are constant and their zero-point contributions can be precomputed and cached. For activation quantization, the overhead is more significant.

The choice depends on the tensor distribution and hardware constraints. This is not a one-size-fits-all decision, and different layers in the same model may benefit from different approaches.

Weights often follow approximately symmetric distributions centered near zero. Neural network initialization schemes like Xavier and Kaiming initialization deliberately produce symmetric distributions, and gradient descent tends to preserve this symmetry for weights in the interior of the network. Symmetric quantization works well here, with minimal range waste and lower computational overhead.

Activations frequently have asymmetric distributions, especially after ReLU (where all values are non-negative) or after certain normalization layers that shift the mean away from zero. Asymmetric quantization better captures these distributions by placing the integer range entirely over the non-zero region.

Hardware support varies considerably. Some accelerators handle symmetric quantization much more efficiently, which makes it preferable even when asymmetric would be theoretically better. The practical speedup depends on whether the hardware has efficient instructions for the zero-point operations. In environments where hardware only accelerates symmetric quantization, choosing asymmetric for its theoretical precision advantage may result in slower inference.

In practice, weight quantization commonly uses symmetric schemes while activation quantization uses asymmetric schemes, though this varies by implementation. The decision ultimately balances mathematical optimality against engineering constraints, and different deployment scenarios may favor different choices. When in doubt, benchmark both approaches on your target hardware with your target model before committing.