GPTQ

Quantise one column, push its error into the columns still to come, weighted by the inverse Hessian: second-order error compensation, column by column.

Loading the animation…

Concept

Rounding each weight to its nearest grid point (round-to-nearest, RTN) minimises the error in the weights. What matters is the error in the layer's output, $\| W X - \hat W X \|^2$ over real inputs $X$ , and there the errors of different weights can cancel. GPTQ (Frantar et al.) quantises a layer one input column at a time and, after each column, adjusts the columns not yet quantised to compensate for the error just made, using second-order information about the inputs: the Hessian $\mathbf{H} = 2 X X^\top$ of the output error.

Watch the animation: a column turns into its quantised values, and the weights to its right shift a little (the middle panel), each by an amount set by how strongly its input is correlated with the one just rounded. The bottom panel tracks the layer's output error with the columns done so far quantised. Plain rounding of the same columns climbs steadily; GPTQ climbs more slowly, because each step's compensation undoes part of the damage. On this layer (8 outputs, 16 inputs, 64 calibration inputs) GPTQ ends at 1.267 against 2.57 for INT4 round-to-nearest, 2.029 times smaller; at INT3, 11.47 against 15.78.

GPTQ's contribution was making this affordable at scale: the paper quantises 175-billion-parameter models to 3 or 4 bits per weight in about four GPU hours. It processes every row in the same column order (which turns out to cost little accuracy), batches the updates so the GPU stays busy, and works from a Cholesky factor of $\mathbf{H}^{-1}$ for numerical stability.

Maths

GPTQ descends from Optimal Brain Quantization (OBQ), itself from Optimal Brain Surgeon. For one output row $w$ with the quadratic loss $(w - \hat w)^\top \mathbf{H} (w - \hat w)/2$ , quantising coordinate $j$ to $q_j$ and letting the free coordinates $F$ re-optimise gives the update

\delta_F = -\frac{w_j - q_j}{[\mathbf{H}_F^{-1}]_{jj}} \, (\mathbf{H}_F^{-1})_{:,j}.

After removing $j$ , the inverse Hessian of the remaining coordinates is one step of Gaussian elimination on $\mathbf{H}_F^{-1}$ . Going through the columns in a fixed order, those successive eliminations are exactly the rows of the upper Cholesky factor $U$ of $\mathbf{H}^{-1}$ , so the whole algorithm needs one Cholesky decomposition, not one matrix update per weight. The dampening term (here 1% of the mean diagonal added to $\mathbf{H}$ ) keeps the factorisation stable when inputs are nearly collinear.

Code

The inner loop, cut from src/lib/num/model.ts (U is the upper Cholesky factor of $\mathbf{H}^{-1}$ ; the scales are fixed per output row from the original weights):

for (let j = 0; j < d; j++) {
  const errs: number[] = [];
  for (let r = 0; r < rows; r++) {
    const w = Wc[r]![j]!;
    const q = dequantInt(quantInt(w, params[r]!), params[r]!);
    Q[r]![j] = q;
    const e = (w - q) / U[j]![j]!;
    errs.push(e);
    for (let k = j + 1; k < d; k++) Wc[r]![k]! -= e * U[j]![k]!;
  }

The Python reference is checked against a numpy implementation of the same algorithm, and this TypeScript port against the Python, bit for bit.