Enable cross-devices Half-Quadratic Quantization for LLMs

[yiliu30]

Type of Change

Feature

Description

Half-Quadratic Quantization (HQQ) Intro:

HQQ requiring no calibration data, significantly speeds up the quantization of large models, while offering compression quality competitive with that of calibration-based methods. For instance, HQQ takes less than 5 minutes to process the colossal Llama-2-70B, that’s over 50x faster compared to the widely adopted GPTQ.

• Add Auto_Accelerator to detect devices automatically.
• Port HQQ from https://github.com/mobiusml/hqq to enable cpu/cuda support
• UTs
• Examples Add HQQ examples #1601

Usage:

from neural_compressor.torch.quantization import get_default_hqq_config, quantize
model = AutoModelForCausalLM.from_pretrained("facebook/opt-125m")
q_model = quantize(model, quant_config=get_default_hqq_config())

How has this PR been tested?

Local test and UTs

Dependency Change?

• numpy
• psutil
• torch