Commit 84469d6

authored

[Inference] Qwen2 support fp8 inference (#8954)

* qwen2 fp8 * fp8 check * fp8 cutlass * int8 cachekv * a8w8c8_fp8

1 parent a275ab7 commit 84469d6Copy full SHA for 84469d6

2 files changed

+451

-55

lines changed

paddlenlp/experimental/transformers/qwen2
- modeling.py
- ptq_fp8_scales_map.json

+451

-55

lines changed

Comments

(0)