Skip to content

Conversation

crazydemo
Copy link
Collaborator

@crazydemo crazydemo commented May 13, 2025

This PR added below test cases:

  1. TestLlama3_3_70BInstruct bfloat16 accuracy test on mmlu and gsm8k.
  2. TestNemotronH-8b bf16 accuracy test on mmlu and gsm8k. (adjust the max_batch_size, due to 5264431
  3. quickstart test case on qwen3 and llm_decoder.

@crazydemo
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4958 [ run ] triggered by Bot

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make it as an accuracy test?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do mmlu check in test_llm_api_pytorch.py. Could you please re-review this PR? There's some changes in accuracy_core.py.

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4958 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #3600 completed with status: 'FAILURE'

@crazydemo crazydemo force-pushed the add_nemotron branch 6 times, most recently from f295233 to d363f00 Compare May 13, 2025 07:30
@crazydemo
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4978 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #4978 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3615 completed with status: 'FAILURE'

@crazydemo
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5094 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5094 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3711 completed with status: 'FAILURE'

@crazydemo
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5132 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5132 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3739 completed with status: 'FAILURE'

@crazydemo
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5251 [ run ] triggered by Bot

@crazydemo
Copy link
Collaborator Author

/bot run --skip_test

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5269 Bot args parsing error: usage: /bot [-h]
{run,kill,skip,submit,reviewers,reuse-pipeline,reuse-review} ...
/bot: error: unrecognized arguments: --skip_test

@LarryXFly LarryXFly merged commit b0ce137 into NVIDIA:main May 15, 2025
2 of 3 checks passed
@tensorrt-cicd
Copy link
Collaborator

PR_Github #5251 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #3837 completed with status: 'FAILURE'

chzblych added a commit that referenced this pull request May 15, 2025
chzblych added a commit that referenced this pull request May 15, 2025
Revert "[test] add qa test mentioned in docs (#4248)"

This reverts commit b0ce137.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants