Integer Data Types in Quantization

After understanding the fundamental difference between floating-point and fixed-point representations, let's focus on the specific integer data types commonly used in model quantization. The core idea is to replace the high-precision floating-point numbers (like 32-bit floats, or $FP32$) used for weights and sometimes activations in the original LLM with lower-precision integers. This drastically reduces the model's memory footprint and can significantly speed up computations, especially on hardware optimized for integer arithmetic.

The most prevalent integer types in quantization are 8-bit integers ($INT8$) and 4-bit integers ($INT4$).

INT8: The Workhorse of Quantization

An 8-bit integer uses 8 bits to represent a number. This allows for $2^8 = 256$ distinct values. Depending on whether we need to represent negative numbers, $INT8$ can be:

• Signed INT8: Represents both positive and negative values. The typical range is from -128 to 127. This is useful for representing weights or activations which often center around zero.
• Unsigned INT8: Represents only non-negative values. The range is from 0 to 255. This might be suitable for activations that are always positive, such as those following a ReLU activation function, although signed integers are more common for weights.

Using $INT8$ reduces the data size by a factor of 4 compared to $FP32$ (since $FP32$ uses 32 bits). This directly translates to a 4x reduction in model size if only weights are quantized, and potentially significant memory savings for activations during inference. Many modern CPUs and GPUs have specialized instructions for performing $INT8$ computations quickly, leading to substantial speedups. $INT8$ quantization often provides a good balance between computational efficiency and maintaining model accuracy, making it a popular choice for many applications.

INT4: Pushing the Limits of Compression

For even greater compression and potential speed gains, 4-bit integers ($INT4$) can be used. A 4-bit integer allows for only $2^4 = 16$ distinct values.

• Signed INT4: Represents values typically in the range -8 to 7.
• Unsigned INT4: Represents values in the range 0 to 15.

Moving from $FP32$ to $INT4$ represents an 8x reduction in data size. This is particularly attractive for deploying very large models on resource-constrained devices or for fitting larger models into available GPU memory. However, representing the original range of floating-point values with only 16 distinct levels is much more challenging than with 256 levels ($INT8$). Consequently, $INT4$ quantization generally introduces more quantization error and can lead to a noticeable drop in model accuracy if not applied carefully. Advanced techniques, which we will cover later (like GPTQ and AWQ in Chapter 3), are often required to achieve good performance with $INT4$.

Other Integer Types

While $INT8$ and $INT4$ are the most common, research and specific applications sometimes explore other bit widths:

• INT16: Offers higher precision than $INT8$ but less compression compared to $FP32$. It's sometimes used as an intermediate step or for quantizing particularly sensitive parts of a model.
• INT3, INT2, Binary (INT1): These represent more extreme forms of quantization, offering maximum compression but typically suffering from significant accuracy degradation. They often require specialized training techniques (like QAT) or hardware support.

The chart below visualizes the reduction in bit width for common data types used in quantization compared to the standard $FP32$.

Comparison of the number of bits used by different numerical data types relevant to model quantization. Lower bit widths lead to smaller model sizes and potentially faster inference.

The choice of integer data type involves a trade-off. Lower bit widths (like $INT4$) provide greater memory savings and potential speedups but increase the risk of accuracy loss due to the coarser representation. Higher bit widths (like $INT8$) retain more precision but offer less compression. The selection depends on the specific LLM, the target hardware, and the acceptable tolerance for performance degradation.

Understanding these integer types is foundational. In the following sections, we'll explore how we map the original floating-point values onto the limited range offered by these integers using different schemes and granularities.