Skip to content

[doc] fix documentation for quantized training #6528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 9, 2024
Merged

[doc] fix documentation for quantized training #6528

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/Parameters.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -680,7 +680,7 @@ Learning Control Parameters

- gradient quantization can accelerate training, with little accuracy drop in most cases

- **Note**: can be used only with ``device_type = cpu``
- **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``

- *New in version 4.0.0*

Expand All @@ -690,7 +690,7 @@ Learning Control Parameters

- with more bins, the quantized training will be closer to full precision training

- **Note**: can be used only with ``device_type = cpu``
- **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``

- *New in 4.0.0*

Expand All @@ -700,14 +700,16 @@ Learning Control Parameters

- renewing is very helpful for good quantized training accuracy for ranking objectives

- **Note**: can be used only with ``device_type = cpu``
- **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``

- *New in 4.0.0*

- ``stochastic_rounding`` :raw-html:`<a id="stochastic_rounding" title="Permalink to this parameter" href="#stochastic_rounding">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool

- whether to use stochastic rounding in gradient quantization

- **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``

- *New in 4.0.0*

IO Parameters
Expand Down
7 changes: 4 additions & 3 deletions include/LightGBM/config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -619,23 +619,24 @@ struct Config {
// desc = enabling this will discretize (quantize) the gradients and hessians into bins of ``num_grad_quant_bins``
// desc = with quantized training, most arithmetics in the training process will be integer operations
// desc = gradient quantization can accelerate training, with little accuracy drop in most cases
// desc = **Note**: can be used only with ``device_type = cpu``
// desc = **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``
// desc = *New in version 4.0.0*
bool use_quantized_grad = false;

// desc = number of bins to quantization gradients and hessians
// desc = with more bins, the quantized training will be closer to full precision training
// desc = **Note**: can be used only with ``device_type = cpu``
// desc = **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``
// desc = *New in 4.0.0*
int num_grad_quant_bins = 4;

// desc = whether to renew the leaf values with original gradients when quantized training
// desc = renewing is very helpful for good quantized training accuracy for ranking objectives
// desc = **Note**: can be used only with ``device_type = cpu``
// desc = **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``
// desc = *New in 4.0.0*
bool quant_train_renew_leaf = false;

// desc = whether to use stochastic rounding in gradient quantization
// desc = **Note**: can be used only with ``device_type = cpu`` and ``device_type=cuda``
// desc = *New in 4.0.0*
bool stochastic_rounding = true;

Expand Down