-
-
Notifications
You must be signed in to change notification settings - Fork 9.2k
Update fp4 quantize API #21327
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update fp4 quantize API #21327
Conversation
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request updates the FlashInfer FP4 quantization API by renaming fp4_swizzle_blockscale
to nvfp4_block_scale_interleave
and updating a function call in flashinfer_cutlass_moe
to use keyword arguments. The code changes are consistent across the modified files and improve readability. A significant concern is the absence of tests to validate these changes. Modifying low-level quantization kernels without corresponding tests introduces a high risk of regressions. It is highly recommended to add unit or integration tests that specifically exercise the FP4 MoE functionality with the updated FlashInfer API to ensure the integration is correct and robust.
27567c7
to
5d52dc4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks for the quick fix!
Signed-off-by: Shu Wang <[email protected]>
Head branch was pushed to by a user without write access
5d52dc4
to
44bb0cf
Compare
Signed-off-by: Shu Wang <[email protected]>
Signed-off-by: Shu Wang <[email protected]> Signed-off-by: qizixi <[email protected]>
Signed-off-by: Shu Wang <[email protected]>
Signed-off-by: Shu Wang <[email protected]> Signed-off-by: avigny <[email protected]>
Signed-off-by: Shu Wang <[email protected]>
Signed-off-by: Shu Wang <[email protected]> Signed-off-by: x22x22 <[email protected]>
Signed-off-by: Shu Wang <[email protected]>
Signed-off-by: Shu Wang <[email protected]>
Update fp4 quantize API
cc. @alexm-redhat
Essential Elements of an Effective PR Description Checklist
supported_models.md
andexamples
for a new model.Purpose
Test Plan
Test Result
(Optional) Documentation Update