How attention sinks keep language models stable(hanlab.mit.edu)191 points by pr337h4m 1 year ago | 31 comments
SVDQuant+NVFP4: 4× Smaller, 3× Faster FLUX with 16-bit Quality on Blackwell GPUs(hanlab.mit.edu)48 points by lmxyy 1 year ago | 8 comments
SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup(hanlab.mit.edu)167 points by lmxyy 1 year ago | 56 comments