Skip to content

[TRTLLM-3330][feat] Support DeepSeek-R1 W4A8 on Hopper #4123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
May 14, 2025
Merged

[TRTLLM-3330][feat] Support DeepSeek-R1 W4A8 on Hopper #4123

merged 5 commits into from
May 14, 2025

Conversation

Barry-Delaney
Copy link
Collaborator

@Barry-Delaney Barry-Delaney commented May 7, 2025 •
edited
Loading

This PR enables W(INT)4-A(FP)8 for DeepSeek-R1 on Hopper.

@Barry-Delaney Barry-Delaney self-assigned this May 7, 2025
@Barry-Delaney
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4377 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4377 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3142 completed with status: 'FAILURE'

@Barry-Delaney
Copy link
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4400 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4400 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3164 completed with status: 'FAILURE'

@Barry-Delaney
Copy link
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4476 [ run ] triggered by Bot

@Barry-Delaney
Copy link
Collaborator Author

/bot kill

@Barry-Delaney
Copy link
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4490 [ kill ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4476 [ run ] completed with state ABORTED

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4490 [ kill ] completed with state SUCCESS
Successfully killed previous jobs for commit ca97639

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4491 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4491 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3226 completed with status: 'SUCCESS'

jhaotingc
jhaotingc reviewed May 8, 2025
View reviewed changes
@Barry-Delaney
Copy link
Collaborator Author

/bot run --disable-fail-fast

@Barry-Delaney
Copy link
Collaborator Author

/bot kill

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4730 [ run ] triggered by Bot

@Barry-Delaney
Address checkpoint update
1d65136 
Signed-off-by: Barry Kang <[email protected]>
@Barry-Delaney
Update README
f659c95 
Signed-off-by: Barry Kang <[email protected]>
@Barry-Delaney
Copy link
Collaborator Author

/bot run

1 similar comment
@Barry-Delaney
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5088 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5088 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3704 completed with status: 'SUCCESS'

@Barry-Delaney
Update version.txt
b27ffef 
Signed-off-by: Barry Kang <[email protected]>
@Barry-Delaney
Copy link
Collaborator Author

/bot skip --comment "Updating version only, skipping tests."

@Barry-Delaney Barry-Delaney enabled auto-merge (squash) May 14, 2025 06:40
@Barry-Delaney
Copy link
Collaborator Author

/bot kill

@Barry-Delaney
Copy link
Collaborator Author

/bot skip --comment "Updating version only, skipping tests."

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5120 [ skip ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5133 [ kill ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5120 [ skip ] completed with state ABORTED

@Barry-Delaney
Copy link
Collaborator Author

/bot kill

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5133 [ kill ] completed with state SUCCESS
Successfully killed previous jobs for commit b27ffef

@Barry-Delaney
Copy link
Collaborator Author

/bot skip --comment "Updating version only, skipping tests"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5137 [ kill ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5137 [ kill ] completed with state SUCCESS
Successfully killed previous jobs for commit b27ffef

@Barry-Delaney
Copy link
Collaborator Author

/bot skip --comment "Updating version only, skipping tests"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5139 [ skip ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5139 [ skip ] completed with state SUCCESS
Skipping testing for commit b27ffef

@Barry-Delaney Barry-Delaney merged commit 20b4291 into NVIDIA:main May 14, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jhaotingc jhaotingc jhaotingc approved these changes

@QiJune QiJune QiJune approved these changes

@xiaoweiw-nv xiaoweiw-nv Awaiting requested review from xiaoweiw-nv

@juney-nvidia juney-nvidia Awaiting requested review from juney-nvidia

@Tracin Tracin Awaiting requested review from Tracin

+1 more reviewer

@StudyingShao StudyingShao StudyingShao left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@Barry-Delaney Barry-Delaney

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Barry-Delaney @tensorrt-cicd @jhaotingc @QiJune @StudyingShao