Skip to content

Conversation

@Yunnglin
Copy link
Collaborator

@Yunnglin Yunnglin requested a review from Copilot June 11, 2025 11:43
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for a new repetition_penalty parameter through the arguments parser and ensures it is sent in the OpenAI API plugin, and updates the CLI tests to reflect expanded generation config options.

  • Introduce repetition_penalty in the Arguments class and CLI parser
  • Map repetition_penalty into the payload in the OpenAI API plugin
  • Update CLI test to use a broader set of sampling parameters

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
tests/cli/test_run.py Expanded --generation-config string to include sampling options
evalscope/perf/plugin/api/openai_api.py Added repetition_penalty field mapping in API payload
evalscope/perf/arguments.py Added repetition_penalty argument and CLI parser option
Comments suppressed due to low confidence (2)

tests/cli/test_run.py:66

  • The new repetition_penalty parameter is not included in this CLI test's generation-config string, so its behavior isn't covered. Consider adding repetition_penalty=<value> to ensure test coverage for this flag.
f'--generation-config do_sample=true,temperature=0.6,max_length=65535,max_new_tokens=65535,max_tokens=65535,n=1,top_p=0.95,top_k=20 ' \

evalscope/perf/plugin/api/openai_api.py:78

  • [nitpick] Consider adding a brief comment or updating external documentation to explain how repetition_penalty influences model output, improving clarity for future maintainers.
if param.repetition_penalty is not None:

@Yunnglin Yunnglin merged commit 502144f into main Jun 12, 2025
1 check passed
yucc-leon pushed a commit to yucc-leon/evalscope that referenced this pull request Oct 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants