AWQ Quant

Activation-aware weight quantization for LLMs — high-quality 4-bit inference.