MACaQ quantizer ships 2-4 bit GGUF and W4A4 from Huawei under Apache 2.0

MACaQ is a calibrated quantizer. It quantizes both weights and activations and uses calibration data, including conditioning images for image-to-video. The paper is on arXiv as 2611.10649, titled "MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations". Its authors are Lorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang and Lukas Cavigelli. The release also includes MACaQ-R, a separate tool that refines GGUF files that are already quantized. MACaQ-R output runs on a standard llama.cpp installation, so it is the relevant piece for teams that already deploy on llama.cpp and do not want to change runtimes.

Each backend supports a different precision. The README says to use Python 3.12 and a separate environment for each backend, so using all four means maintaining four environments.

The published llama.cpp recipes are mostly Qwen: - dense Qwen3-4B - hybrid Qwen3.8-27B - the MoE models Qwen3-30B-A3B, Qwen3-Coder-30B-A3B-Instruct, Qwen3.6-35B-A3B and Qwen3-Next-80B-A3B-Instruct - Qwen3.8-Flash-Next, which has a model-specific recipe The only non-Qwen text recipe is allenai/OLMoE-1B-7B-0125-Instruct. The Diffusers recipes cover FLUX.2-klein-4B, Wan2.2-TI2V-5B-Diffusers and stable-audio-1.0. The PyTorch recipes cover Qwen3-TTS and Chatterbox Turbo. The project calls itself model-agnostic, but the README publishes no Llama, Mistral, Gemma or DeepSeek GGUF workflow, so quality on those architectures is not shown in the repository.

The README makes two claims against MR-GPTQ and DuQuant++. First, MACaQ reaches lower perplexity than both. Second, it can match their best Llama-3-8B perplexity with 11 times less quantization time. The README gives neither the Llama-3-8B perplexity values nor the bit configuration used, and points to the paper for details. It does not compare against plain GPTQ and does not mention AWQ. Teams using AWQ as their baseline get no direct number from this release and will need to run their own evaluation.

The README's numeric comparison with existing llama.cpp quants is for MACaQ-R. MACaQ-R refines the weight codes inside an already-quantized GGUF and preserves its native formats and block scales, so the output runs on a standard llama.cpp installation. On Qwen3.6-35B-A3B the Q4_K_XL file stays at 22.36 GB. KL divergence to FP16 drops from 0.015731 for Unsloth's UD-Q4_K_XL to 0.013367, about 15 percent lower. Separately, the README offers a comparison of MACaQ itself with published Qwen3-4B GGUFs by file size and KL divergence to FP16, but its text gives no numbers and points to the MACaQ blog for details of the models compared. No downstream task accuracy is reported in the README for either model.

MACaQ-R takes four inputs: - the quantized GGUF - the original Hugging Face weights - an FP16 GGUF teacher - a calibration text file The README describes the input as a quantized MoE GGUF and does not say whether dense models are supported.

There is a distinction to watch. MACaQ-R files run on stock llama.cpp, but the full MACaQ workflow with activation transforms uses the runtime bundled in the repo. Commenters on Hacker News pointed to people on r/LocalLLaMA reproducing the method and publishing GGUFs. One commenter called the results much better than those of the previously widely used quantizers. That is community reaction, not a controlled benchmark.

On the quantization side, the README reports a 122B-parameter model quantized in 64 minutes using 32.5 GB of device memory. It does not name the 122B model or the accelerator used. That figure is memory used during quantization, not the inference footprint. The README gives no memory comparison with GPTQ, AWQ or llama.cpp quants, and its only cost comparison is the 11x reduction in quantization time against MR-GPTQ and DuQuant++.

On the inference side, MACaQ W4A4 on Qwen3.8-27B in vLLM reaches a perplexity of 2.8994, against 2.8463 for BF16. Relative to BF16, prefill is 2.33 times faster and decode is 2.58 times faster. The README does not give: - the GPU that produced those speedups - the batch size or sequence length - the dataset the perplexity was measured on Those details are needed before assuming the speedup transfers to other serving hardware.

For Huawei's own accelerators, MACaQ W4A4 dense and grouped matmul kernels are in two CANN pull requests: - ops-nn #12568 adds PTO-ISA kernels via ascend_ops_nn. - ops-tilelang #44 adds TileLang kernels with the CCE backend. The README does not say whether these have been merged. It says full-model execution requires integration with an Ascend inference runtime, so Ascend currently has kernels, not an end-to-end serving path. The README does not name the GPUs the vLLM backend was tested on. The llama.cpp backend supports CPU execution, and MACaQ-R files run on a standard llama.cpp installation, which makes CPU-only llama.cpp the most portable target in the release.

MACaQ is licensed under the Apache License 2.0, and bundled third-party components retain their own licenses. The repository includes a NOTICE file, which is worth checking before vendoring the code. At the time of the snapshot the repo had 182 stars, 7 forks and 22 commits. One Hacker News commenter worried that Huawei-hosted pages outside China could disappear because of sanctions. Another replied that many of the organisation's GitHub repos have been around for a while. Under Apache 2.0, the code can be forked and kept.

Commenters also asked why Huawei's three most-starred GitHub repositories are all quantizers. One replied that the three do different jobs: - SINQ, created last year, is weight-only and calibration-free. - KVarN quantizes the KV cache to get longer context. - MACaQ is multimodal, calibrated quantization of both weights and activations. That split comes from a commenter, not from the README. It is consistent with what the repository shows, where W4A4 activation quantization and calibration data are central.