Skip to content

[Feature] Add A New Approach of Dataset Integration based on ChatML Template with Eval Examples #2277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Oct 14, 2025
Merged

[Feature] Add A New Approach of Dataset Integration based on ChatML Template with Eval Examples #2277

merged 10 commits into from
Oct 14, 2025

Conversation

@Myhs-phz
Copy link
Collaborator

WIP

@Myhs-phz Myhs-phz added the WIP label Sep 18, 2025
@Myhs-phz Myhs-phz changed the title [Feature] Add chat_obj_custom dataset config [Feature] Add ChatMLDataset and Chat_Obj_Custom Dataset Config Sep 25, 2025
@Myhs-phz Myhs-phz removed the WIP label Sep 29, 2025
tonysy
tonysy requested changes Sep 30, 2025
View reviewed changes
@tonysy
Copy link
Collaborator

tonysy commented Sep 30, 2025

Also provide a doc on how to use the chatmldataset(via config or CLI), and how to add your own chatmldatset.
We hope such a major feature will ease the support of new benchmark or the user can evaluate their dataset without modfiy the source code(only use the opencompass).

@Myhs-phz Myhs-phz changed the title [Feature] Add ChatMLDataset and Chat_Obj_Custom Dataset Config [Feature] Add A New Approach of Dataset Integration based on ChatML Template with Eval Examples Sep 30, 2025
@Myhs-phz Myhs-phz requested a review from tonysy October 10, 2025 06:03
zhulinJulia24
zhulinJulia24 approved these changes Oct 14, 2025
View reviewed changes
Copy link
Collaborator

@zhulinJulia24 zhulinJulia24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tonysy
tonysy approved these changes Oct 14, 2025
View reviewed changes
Copy link
Collaborator

@tonysy tonysy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Myhs-phz Myhs-phz merged commit 8ee07ac into open-compass:main Oct 14, 2025
7 of 8 checks passed
zyc140345 pushed a commit to zyc140345/opencompass that referenced this pull request Oct 23, 2025
@Myhs-phz
[Feature] Add A New Approach of Dataset Integration based on ChatML T…
c567e1e 
…emplate with Eval Examples (open-compass#2277)

* test

* add feature

* fix

* add chatml dataset

* fix

* fix

* fix

* add datasets, example and doc

* fix

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tonysy tonysy tonysy approved these changes

@zhulinJulia24 zhulinJulia24 zhulinJulia24 approved these changes

Assignees

@acylam acylam

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Myhs-phz @tonysy @zhulinJulia24 @acylam