AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization

(nvidia.github.io)

2 points | by khalidkhair 6 hours ago ago