-
Notifications
You must be signed in to change notification settings - Fork 285
Print the scheduler_config info #2467
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Print the scheduler_config info #2467
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please add test to show that it is printed out in debug mode and not printed, otherwise
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please add tests? Thanks
…genai into guozhong/print_scheduler_config_info
Yes. Will add. |
@rkazants Is there any existing test case I can refer to? Thanks |
…genai into guozhong/print_scheduler_config_info
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds debug logging capability for scheduler configuration information when the OPENVINO_LOG_LEVEL environment variable is set. It implements to_string()
methods for various configuration classes and integrates scheduler config printing into the ContinuousBatchingPipeline constructors.
Key changes:
- Added
to_string()
methods to SchedulerConfig, SparseAttentionConfig, CacheEvictionConfig, and KVCrushConfig classes - Added utility function
print_scheduler_config_info()
to conditionally print scheduler configuration - Integrated scheduler config printing into all ContinuousBatchingPipeline constructor paths
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
File | Description |
---|---|
src/python/py_continuous_batching_pipeline.cpp | Exposed new to_string() methods to Python bindings |
src/cpp/src/utils.hpp | Added declaration for print_scheduler_config_info() function |
src/cpp/src/utils.cpp | Implemented print_scheduler_config_info() function |
src/cpp/src/continuous_batching/pipeline.cpp | Added scheduler config printing calls in all constructor paths |
src/cpp/include/openvino/genai/sparse_attention.hpp | Added to_string() method to SparseAttentionConfig class |
src/cpp/include/openvino/genai/scheduler_config.hpp | Added to_string() method to SchedulerConfig class |
src/cpp/include/openvino/genai/cache_eviction.hpp | Added to_string() methods to KVCrushConfig and CacheEvictionConfig classes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
All tests are now executed with enabled logging: #2621 |
…genai into guozhong/print_scheduler_config_info
Added test case, pls review again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
When setting OPENVINO_LOG_LEVEL, print the info of scheduler_config.
SchedulerConfig {
max_num_batched_tokens: 18446744073709551615
num_kv_blocks: 0
cache_size: 0
dynamic_split_fuse: 1
use_cache_eviction: 0
max_num_seqs: 256
enable_prefix_caching: 0
use_sparse_attention: 0
}