Search results

Can we use W8A8B8O8Linear in LLaMA model?

Status: Open.

#26 In AniZpZ/AutoSmoothQuant;

· peilin-chen opened on Oct 2, 2024

llama2-7b-chat量化完推理报错

Status: Open.

#24 In AniZpZ/AutoSmoothQuant;

· AlexMa0 opened on Jul 9, 2024

是否支持int4量化

Status: Open.

#23 In AniZpZ/AutoSmoothQuant;

· AlexMa0 opened on Jun 28, 2024

When quantizing baichuan2-7B, I encountered an error: AttributeError: 'RMSNorm' object has no attribute 'epsilon'.

Status: Open.

#20 In AniZpZ/AutoSmoothQuant;

· Flyipig opened on Apr 24, 2024

name 'position_ids' is not defined

Status: Open.

#16 In AniZpZ/AutoSmoothQuant;

· LMX-xin opened on Mar 13, 2024

Tensor shape error when loading quant Llama-2-70B

Status: Open.

#14 In AniZpZ/AutoSmoothQuant;

· MingLin-home opened on Feb 29, 2024

Any plan for supporting Mistral 7B model?

Status: Open.

#13 In AniZpZ/AutoSmoothQuant;

· leocnj opened on Feb 28, 2024

Cannot quantize to int8 - torch TypeError

Status: Open.

#11 In AniZpZ/AutoSmoothQuant;

· AlpinDale opened on Feb 3, 2024

Difference between W8A8BFP32OFP32LinearWithQuantScale and W8A8BFP32OFP32Linear

Status: Open.

#10 In AniZpZ/AutoSmoothQuant;

· Hongbosherlock opened on Feb 2, 2024

Question about per-token quant

Status: Open.

#9 In AniZpZ/AutoSmoothQuant;

· Hongbosherlock opened on Feb 2, 2024

量化后推理速度

efficiencyInference speed or throughputInference speed or throughput

Status: Open.

#4 In AniZpZ/AutoSmoothQuant;

· jundolc opened on Jan 26, 2024