INT8 Quantization › The INT8 Representation Space

An 8-bit signed integer can stand for values from -128 to 127, giving us exactly 256 distinct values to work with. Think of this as having 256 labeled slots on a number line, and every floating-point weight or activation value must be assigned to the nearest slot. When we quantize a floating-point tensor, we create a mapping (technically a codebook) that associates each of these 256 integers with a specific floating-point value. The integers become the compact representation we store and compute with, and the codebook lets us convert back to floating-point whenever we need the full precision.

To understand this mapping intuitively, imagine you have a number line representing all possible floating-point values a weight might take. Quantization places 256 evenly-spaced "buckets" along this line, and every floating-point value must be assigned to its nearest bucket. The original value is then represented by that bucket's integer label, and reconstruction involves converting the label back to the bucket's center position. The spacing between these buckets, and where they are positioned along the number line, completely determines the quality of our approximation. Place them well and every value lands close to its bucket center. Place them poorly and some values are forced into buckets far from where they belong.

The basic challenge is choosing how to position this grid. Consider a weight tensor with values ranging from -0.5 to 1.2. We need to decide what floating-point value should -128 stand for, what floating-point value should 127 stand for, and how to handle values outside our chosen range (clipping them to the boundary). These choices define our quantization scheme and directly impact the reconstruction error. A poorly chosen grid might waste many of its 256 integers representing values that never occur in the tensor, while cramming the observed values into too few buckets. The goal is to align our quantization grid as closely as possible with the distribution of values we need to stand for.

The key insight is that the quality of quantization depends almost entirely on how well the grid matches the distribution of values in the tensor. If our weights happen to be uniformly distributed between -1 and 1, a grid spanning that range is nearly optimal. But if most weights cluster near zero with a long tail of larger values, we need to make a choice: use a wide grid that accommodates the tail but has poor resolution near zero, or use a narrow grid that captures the bulk well but clips the tail. Neither choice is free, and the right answer depends on which values matter more for the model's accuracy.