Enable FA V3

[micmelesse]

enable FA V3. It adds support for f8. We also add support for Paged Attention.

Build Flash attention v3

cd hooper
FLASH_ATTENTION_TRITON_AMD_ENABLE="TRUE" python setup.py install

You can use fp8 like this after installing flash attention v3. Make sure that the environment variable FLASH_ATTENTION_TRITON_AMD_ENABLE="TRUE" is set

from flash_attn_interface import flash_attn_func

out = flash_attn_func(
    q,
    k,
    v,
    q_descale=q_descale,
    k_descale=k_descale,
    v_descale=v_descale
)

If you want use paged attention with fp8 you can use the flash_attn_with_kvcache api

from flash_attn_interface import flash_attn_with_kvcache

out = flash_attn_with_kvcache(
    q,
    k_cache,
    v_cache,
    q_descale=q_descale,
    k_descale=k_descale,
    v_descale=v_descale,
    page_table=page_table,
)

[micmelesse]

fa v2 tests. increased tests to 100k

fa v2 bench

fa v3 tests.