You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/proposals/parameter-distribution.md
+99-45Lines changed: 99 additions & 45 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,47 +4,119 @@
4
4
The goal of this project is to enhance the existing Katib Experiment APIs to support various parameter distributions such as uniform, log-uniform, and qlog-uniform. Then extend the suggestion services to be able to configure distributions for search space using libraries provided in each framework.
5
5
6
6
## Motivation
7
-
Currently, Katib is limited to supporting only uniform distribution for integer, float, and categorical hyperparameters. By introducing additional distributions, Katib will become more flexible and powerful in conducting hyperparameter optimization tasks.
7
+
Currently, [Katib](https://github.com/kubeflow/katib) is limited to supporting only uniform distribution for integer, float, and categorical hyperparameters. By introducing additional distributions, Katib will become more flexible and powerful in conducting hyperparameter optimization tasks.
8
+
9
+
A Data Scientist requires Katib to support multiple hyperparameter distributions, such as log-uniform, normal, and log-normal, in addition to the existing uniform distribution. This enhancement is crucial for more flexible and precise hyperparameter optimization. For instance, learning rates often benefit from a log-uniform distribution because small values can significantly impact performance. Similarly, normal distributions are useful for parameters that are expected to vary around a central value.
10
+
11
+
### Goals
12
+
- Add `Distribution` field to `ParameterSpec` alongside `ParameterType`.
13
+
- Support for the log-uniform, normal, and log-normal Distributions.
14
+
- Update the Experiment and gRPC API to support `Distribution`.
15
+
- Update logic to handle the new parameter distributions for each suggestion service (e.g., Optuna, Hyperopt).
16
+
- Extend the Python SDK to support the new `Distribution` field.
17
+
### Non-Goals
18
+
- This proposal do not aim to create new version for CRD APIs.
19
+
- No changes will be made to the core optimization algorithms beyond supporting new distributions.
8
20
9
21
## Proposal
10
22
11
-
### Maintaining two versions of CRD APIs
12
-
- Introduce `v1beta2` API version
13
-
- The (conversion webhook)[https://book.kubebuilder.io/multiversion-tutorial/conversion.html] will serve as a critical component in managing transitions between different API versions (`v1beta1` to `v1beta2`).
14
-
- TODO
15
-
> I will create a correspondence table between v1beta1 and v1beta2. Maybe, we only need to create a table for the `ParameterType` and the `FeasibleSpace`.
16
-
Look into this.
17
-
- A specific field will be added to the katib-config `katib/pkg/util/v1beta1/katibconfig/config.go` to handle different gRPC client versions required by the suggestion controller. (One approach could be this for maintaining two versions of gRPC APIs).
- The `Step` field can be used to define quantization steps for uniform or log-uniform distributions, effectively covering q-quantization requirements.
81
+
```go
82
+
typeFeasibleSpacestruct {
83
+
Maxstring`json:"max,omitempty"`
84
+
Minstring`json:"min,omitempty"`
85
+
List []string`json:"list,omitempty"`
86
+
Stepstring`json:"step,omitempty"`// Step can be used to define q-quantization
87
+
}
88
+
```
46
89
47
-
### Suggestion Service Logic
90
+
## gRPC API changes
91
+
Scope: `pkg/apis/manager/v1beta1/api.proto`
92
+
- Add the `Distribution` field to the `ParameterSpec` message
93
+
```
94
+
/**
95
+
* Config for a hyperparameter.
96
+
* Katib will create each Hyper parameter from this config.
97
+
*/
98
+
message ParameterSpec {
99
+
string name = 1; /// Name of the parameter.
100
+
ParameterType parameter_type = 2; /// Type of the parameter.
101
+
FeasibleSpace feasible_space = 3; /// FeasibleSpace for the parameter.
102
+
Distribution distribution = 4; // Distribution of the parameter.
103
+
}
104
+
```
105
+
- Define the `Distribution` enum
106
+
```
107
+
/**
108
+
* Distribution types for HyperParameter.
109
+
*/
110
+
enum Distribution {
111
+
CATEGORICAL = 0;
112
+
UNIFORM = 1;
113
+
LOG_UNIFORM = 2;
114
+
NORMAL = 3;
115
+
LOG_NORMAL = 4;
116
+
}
117
+
```
118
+
119
+
## Suggestion Service Logic
48
120
- For each suggestion service (e.g., Optuna, Hyperopt), the logic will be updated to handle the new parameter distributions.
49
121
- This involves modifying the conversion functions to map Katib distributions to the corresponding framework-specific distributions.
0 commit comments