Fix kv_cache_dtype handling for out-of-tree HPU plugin

[kzawora-intel]

PR #21131 removed HPU checks for --kv-cache-dtype flag, and now HPU out-of-tree plugin (https://github.com/vllm-project/vllm-gaudi) is unable to use KV cache quantization with INC due to NotImplementedError: VLLM_USE_V1=1 is not supported with --kv-cache-dtype.. This PR adds a check for out-of-tree HPU plugin and allows it to use fp8_inc KV cache quantization.

[gemini-code-assist]

Code Review

This pull request fixes an issue with kv_cache_dtype handling for the out-of-tree HPU plugin by adding a specific check to allow fp8_inc quantization. The change correctly enables the feature for HPU. My review includes a suggestion to refactor the added logic for better maintainability and to prevent potential future bugs by integrating the new platform check into the existing conditional block.

[kzawora-intel]

@xuechendi please take a look

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[DarkLight1337]

No need to check auto since this method is only called otherwise

[xuechendi]

@WoosukKwon @mgoin , please help to review.
HPU V1 now supports kv_cache_dtype, while existing codes doesn't provide an assumption that oot device supports kv_cache_dtype.
We want to update that. Thanks

[DarkLight1337]

The default should be something like

            fp8_attention = self.kv_cache_dtype.startswith("fp8")
            will_use_fa = envs.VLLM_ATTENTION_BACKEND == "FLASH_ATTN_VLLM_V1"
            if fp8_attention and will_use_fa:
                from vllm.attention.utils.fa_utils import (
                    flash_attn_supports_fp8)
                return flash_attn_supports_fp8()
            return False

[mgoin]

Why is this so? This logic is specific to CUDA anyway

[DarkLight1337]

I am just following the previous logic strictly.

[DarkLight1337]

Feel free to update if the old logic is wrong

[mgoin]

I think the base class should always return false so it is up to the specific impls to define

Suggested change

	import vllm.envs as envs
	fp8_attention = kv_cache_dtype.startswith("fp8")
	will_use_fa = envs.VLLM_ATTENTION_BACKEND == "FLASH_ATTN_VLLM_V1"
	if fp8_attention and will_use_fa:
	from vllm.attention.utils.fa_utils import flash_attn_supports_fp8
	return flash_attn_supports_fp8()
	return False
	return False

[xuechendi]

@mgoin. Have updated codes based your suggestion