Huawei releases MACaQ, its internal quantizer for local inference(github.com)
121 points by thefox96 | 30 comments
GitHub repository for Huawei CSL's MACaQ, a model-agnostic quantization tool for text, image, video and audio models targeting llama.cpp, vLLM, Diffusers and PyTorch.