What is the GPTQ algorithm?

GPTQ is a one-shot post-training quantization method for the weights of a linear layer. Instead of rounding every weight independently to its nearest low-bit grid point, it quantizes the weight matrix one column at a time, and after each column it adjusts all the not-yet-quantized columns to cancel the error it just committed — so the layer's output W·X stays as close as possible to the original, not the weights themselves. The correction that does this comes from second-order information: the inverse of the Hessian of the layer-output loss. The original paper reports quantizing 175-billion-parameter models to 3–4 bits in about four GPU-hours with negligible accuracy loss, which is why GPTQ became a default for weight-only int4.

Why is the Hessian in GPTQ just X @ X.T?

Because the loss GPTQ minimizes is the squared error of the layer output, L = ||W·X − W_q·X||², and that loss is a quadratic function of the weight error ΔW = W − W_q. Expand it as a trace — ||ΔW·X||² = Trace(ΔW · X·Xᵀ · ΔWᵀ) — and the constant matrix sitting in the middle is X·Xᵀ. The second derivative of a quadratic form is exactly twice that middle matrix, and the factor of 2 is dropped by convention. So the “Hessian” needs no autodiff at all: you run calibration data through the layer, accumulate X·Xᵀ, and you have it. It is also what makes GPTQ data-aware — columns of W that meet large activations get large Hessian entries, and errors there cost more.

What does percdamp (dampening) do in GPTQ?

It adds a small constant — percdamp times the mean of the Hessian diagonal, 1% by default — to every diagonal entry of H before factorizing it. Calibration Hessians are routinely ill-conditioned or outright singular: dead input features give zero rows, correlated features make columns linearly dependent, and too few calibration samples make H rank-deficient by construction. A Cholesky factorization of such a matrix hits a zero or negative pivot and fails, and even near-singular pivots make the error-compensation term (w − dq)/d explode. The damping floor keeps every pivot healthy at the cost of slightly blurring the second-order information — measurably higher quantization loss, but a factorization that always succeeds.

Why does GPTQ process the weight matrix in blocks of 128 columns?

Purely for efficiency — the blocksize does not change the result. Inside a block, each quantized column immediately updates the remaining columns of that block (a rank-one outer-product update). Columns beyond the block are left untouched until the block finishes, at which point all of the block's accumulated errors are applied to the rest of the matrix in one matrix multiply, Err1 @ Hinv[block, rest]. That “lazy batching” turns thousands of thin rank-one updates on a huge matrix into a few large matmuls, which is the difference between bandwidth-bound crawling and compute-bound speed on an accelerator. In exact arithmetic the compensation applied is identical either way, which is why the docstring says the blocksize “should usually not be changed.”

Why does GPTQ use Cholesky factorizations instead of inverting the Hessian directly?

The algorithm needs rows of the inverse Hessian, in column order, and it needs them numerically stable. The qwix implementation gets them in three steps: factor H into a lower-triangular L (Cholesky), turn that factorization into H⁻¹ via cho_solve against the identity (cheaper and stabler than a general inverse), then take the upper Cholesky factor of H⁻¹. That final triangular matrix is exactly what the per-column loop consumes: its diagonal entry d is the “how much does an error here cost” denominator, and its row segment Hinv1[i, i:] is the direction the compensation update pushes the remaining columns. The original GPTQ paper introduced this Cholesky reformulation because accumulating floating-point updates into an explicitly inverted H drifts into numerical breakdown at scale.

Is the qwix gptq_core.py implementation the same as the original GPTQ?

It is a faithful JAX port of the algorithm, and it says so: the module docstring links the paper and the original PyTorch repository, and the code deliberately keeps the same variable names — W, H, Hinv1, Err1, Losses1, blocksize, percdamp — down to a pylint exemption for the naming style. The differences are host-language idioms, not math: jax.new_ref buffers stand in for in-place tensor writes, the scale/zero-point machinery delegates to qwix's own qarray calibration (which supports per-channel and grouped quantization), and the result comes back as a qwix QArray rather than raw integers.

GPTQ, Line by Line: Hessian-Based Weight Quantization

Round a weight to fewer bits and you commit an error — GPTQ's move is to make the weights you have not rounded yet absorb it. We walk Google's real JAX implementation (qwix's gptq_core.py, 220 lines) top to bottom: why the Hessian is just X·Xᵀ, the dampening that keeps a Cholesky alive, the three-line factorization dance that replaces a matrix inverse, and the two nested loops where each column's error becomes a rank-one update on everything to its right. Then the algorithm runs live — scrub the blocksize and damping and watch the compensation wave move.

Code walk · AI / ML. The source ↗

A free, interactive, animated visual explainer of GPTQ, Line by Line: Hessian-Based Weight Quantization — built to be understood, not skimmed.

Questions

What is the GPTQ algorithm?: GPTQ is a one-shot post-training quantization method for the weights of a linear layer. Instead of rounding every weight independently to its nearest low-bit grid point, it quantizes the weight matrix one column at a time, and after each column it adjusts all the not-yet-quantized columns to cancel the error it just committed — so the layer's output W·X stays as close as possible to the original, not the weights themselves. The correction that does this comes from second-order information: the inverse of the Hessian of the layer-output loss. The original paper reports quantizing 175-billion-parameter models to 3–4 bits in about four GPU-hours with negligible accuracy loss, which is why GPTQ became a default for weight-only int4.
Why is the Hessian in GPTQ just X @ X.T?: Because the loss GPTQ minimizes is the squared error of the layer output, L = ||W·X − W_q·X||², and that loss is a quadratic function of the weight error ΔW = W − W_q. Expand it as a trace — ||ΔW·X||² = Trace(ΔW · X·Xᵀ · ΔWᵀ) — and the constant matrix sitting in the middle is X·Xᵀ. The second derivative of a quadratic form is exactly twice that middle matrix, and the factor of 2 is dropped by convention. So the “Hessian” needs no autodiff at all: you run calibration data through the layer, accumulate X·Xᵀ, and you have it. It is also what makes GPTQ data-aware — columns of W that meet large activations get large Hessian entries, and errors there cost more.
What does percdamp (dampening) do in GPTQ?: It adds a small constant — percdamp times the mean of the Hessian diagonal, 1% by default — to every diagonal entry of H before factorizing it. Calibration Hessians are routinely ill-conditioned or outright singular: dead input features give zero rows, correlated features make columns linearly dependent, and too few calibration samples make H rank-deficient by construction. A Cholesky factorization of such a matrix hits a zero or negative pivot and fails, and even near-singular pivots make the error-compensation term (w − dq)/d explode. The damping floor keeps every pivot healthy at the cost of slightly blurring the second-order information — measurably higher quantization loss, but a factorization that always succeeds.
Why does GPTQ process the weight matrix in blocks of 128 columns?: Purely for efficiency — the blocksize does not change the result. Inside a block, each quantized column immediately updates the remaining columns of that block (a rank-one outer-product update). Columns beyond the block are left untouched until the block finishes, at which point all of the block's accumulated errors are applied to the rest of the matrix in one matrix multiply, Err1 @ Hinv[block, rest]. That “lazy batching” turns thousands of thin rank-one updates on a huge matrix into a few large matmuls, which is the difference between bandwidth-bound crawling and compute-bound speed on an accelerator. In exact arithmetic the compensation applied is identical either way, which is why the docstring says the blocksize “should usually not be changed.”
Why does GPTQ use Cholesky factorizations instead of inverting the Hessian directly?: The algorithm needs rows of the inverse Hessian, in column order, and it needs them numerically stable. The qwix implementation gets them in three steps: factor H into a lower-triangular L (Cholesky), turn that factorization into H⁻¹ via cho_solve against the identity (cheaper and stabler than a general inverse), then take the upper Cholesky factor of H⁻¹. That final triangular matrix is exactly what the per-column loop consumes: its diagonal entry d is the “how much does an error here cost” denominator, and its row segment Hinv1[i, i:] is the direction the compensation update pushes the remaining columns. The original GPTQ paper introduced this Cholesky reformulation because accumulating floating-point updates into an explicitly inverted H drifts into numerical breakdown at scale.
Is the qwix gptq_core.py implementation the same as the original GPTQ?: It is a faithful JAX port of the algorithm, and it says so: the module docstring links the paper and the original PyTorch repository, and the code deliberately keeps the same variable names — W, H, Hinv1, Err1, Losses1, blocksize, percdamp — down to a pylint exemption for the naming style. The differences are host-language idioms, not math: jax.new_ref buffers stand in for in-place tensor writes, the scale/zero-point machinery delegates to qwix's own qarray calibration (which supports per-channel and grouped quantization), and the result comes back as a qwix QArray rather than raw integers.

GPTQ, Line by Line: Hessian-Based Weight Quantization

Questions

Related explainers