Support int8 KVCache Quant in Vllm#1507

Closed

AniZpZ wants to merge 57 commits intovllm-project:mainfrom

AniZpZ:kv_quant

Commits on Oct 30, 2023

support kv cache quantization

Lin Pengyun
authored and
AniZpZ
committed
fix python code

Lin Pengyun
authored and
AniZpZ
committed
merge and reformat

Lin Pengyun
authored and
AniZpZ
committed
support generating kv quant parameters and evaluting kv quant models

Lin Pengyun
authored and
AniZpZ
committed
modify test functions

Lin Pengyun
authored and
AniZpZ
committed
fix test code

Lin Pengyun
authored and
AniZpZ
committed
fix test attention
AniZpZ
committed
modify attention kernel test using pytest

Lin Pengyun
authored and
AniZpZ
committed
fix quant parameter passing

Lin Pengyun
authored and
AniZpZ
committed
code clean
AniZpZ
committed
code clean
AniZpZ
committed

Commits on Nov 2, 2023

Merge branch 'main' into kv_quant
AniZpZ
authored

Commits on Nov 3, 2023

code format
AniZpZ
committed
code format
AniZpZ
committed

Commits on Nov 15, 2023

fix merge
AniZpZ
committed

Commits on Nov 20, 2023

fix reshape_and_cache_quantized
AniZpZ
committed

Commits on Nov 22, 2023

tmp fix
AniZpZ
committed
tmp fix2
AniZpZ
committed

Commits on Nov 23, 2023

update kv-quant kernels
zhangying169
committed
add kv-quant kernel tests
zhangying169
committed
support kv-quant
zhangying169
committed

Commits on Nov 24, 2023

code format
AniZpZ
committed
fix work bugs
zhangying169
committed

Commits on Nov 27, 2023

fix unit test
AniZpZ
committed

Commits on Nov 29, 2023

fix unit test
AniZpZ
committed

Commits on Dec 5, 2023

fix kv-quant args
zhangying169
committed

Commits on Dec 18, 2023

fix attention params
zhangying169
committed

Commits on Jan 16, 2024

Merge tag 'v0.2.7' into kv_quant_v0.2.7
zhangying169
committed
format code
zhangying169
committed
add .buildkite
zhangying169
committed

Commits on Feb 4, 2024

merge with remote branch 'vllm/main'
zhangying169
committed

Commits on Feb 5, 2024

Merge branch 'kv_quant_merge' into kv_quant
zhangying169
committed
Merge pull request #13 in wm_ai/project_v from tmp to kv_quant - <merge-MERGE #PR-13 ~merge with remote branch 'vllm/main'
AniZpZ
committed
Merge branch 'main' into kv_quant
AniZpZ
authored
fix compile issue
zhangying169
committed
fix unit test issue
zhangying169
committed

Commits on Feb 7, 2024

fix issues
zhangying169
committed
support exporting kv quant params for transformers>=4.36.0
zhangying169
committed
fix benchmarks for kv cache int8
zhangying169
committed
Merge branch 'main' into kv_quant
HandH1998
authored
fix supporting kv cache int8 for specified models
zhangying169
committed
add int8_kv_cache.rst
zhangying169
committed

Commits on Feb 8, 2024

code format
AniZpZ
committed
code format
AniZpZ
committed

Commits on Feb 19, 2024

code format
AniZpZ
committed
code format
AniZpZ
committed
modify int8 kv cache doc
zhangying169
committed

Commits on Mar 25, 2024

Commits on Mar 26, 2024