Skip to content

Commit 1492ebf

Browse files
shashank-iitbhushashank-iitbhu
committed
API design added
Signed-off-by: Shashank Mittal <[email protected]>
1 parent d853fa5 commit 1492ebf

File tree

1 file changed

+88
-2
lines changed

1 file changed

+88
-2
lines changed

‎docs/proposals/parameter-distribution.md‎

Lines changed: 88 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,10 @@ A Data Scientist requires Katib to support multiple hyperparameter distributions
2525
|-------------------------------|-----------------------|--------------------------------------|-----------------------|
2626
| **Uniform Continuous** | `hp.uniform` | `FloatDistribution` | `tune.uniform` |
2727
| **Quantized Uniform** | `hp.quniform` | `DiscreteUniformDistribution`(deprecated) Use `FloatDistribution` instead. | `tune.quniform` |
28-
| **Log Uniform** | `hp.loguniform` | `LogUniformDistribution`(deprecated) Use `FloatDistribution` instead. | `tune.loguniform` |
28+
| **Log Uniform** | `hp.loguniform` | `LogUniformDistribution`(deprecated) Use `FloatDistribution` instead with `log` boolean. | `tune.loguniform` |
2929
| **Uniform Integer** | `hp.randint` or quantized distributions with step size `q` set to 1 | `IntDistribution` | `tune.randint` |
3030
| **Categorical** | `hp.choice` | `CategoricalDistribution` | `tune.choice` |
31-
| **Quantized Log Uniform** | `hp.qloguniform` | Custom Implementation Use `FloatDistribution` instead. | `tune.qloguniform` |
31+
| **Quantized Log Uniform** | `hp.qloguniform` | Custom Implementation Use `FloatDistribution` instead with `log` boolean step must be `None` if log is true. | `tune.qloguniform` |
3232
| **Normal** | `hp.normal` | (Not directly supported) | `tune.randn` |
3333
| **Quantized Normal** | `hp.qnormal` | (Not directly supported) | `tune.qrandn` |
3434
| **Log Normal** | `hp.lognormal` | (Not directly supported) | (Use custom transformation in `tune.randn`) |
@@ -38,6 +38,7 @@ A Data Scientist requires Katib to support multiple hyperparameter distributions
3838

3939
### How is Nevergrad implementing Hyperopt?
4040
Nevergrad maps parameter types (like p.Scalar, p.Log, p.Choice, etc.) from Nevergrad to corresponding Hyperopt search space definitions (hp.uniform, hp.loguniform, hp.choice, etc.).
41+
ref: https://github.com/facebookresearch/nevergrad/blob/c23f20406d9223afbe253f9807ca7e8ba993a1ac/nevergrad/optimization/externalbo.py#L47
4142
```python
4243
def _get_search_space(param_name, param):
4344
if isinstance(param, p.Scalar):
@@ -52,6 +53,91 @@ def _get_search_space(param_name, param):
5253
```
5354
The `_get_search_space` function constructs a search space that represents the entire parameter space defined by Nevergrad.
5455

56+
## API Design
57+
- [Distribution](#api-v1-beta1-Distribution)
58+
- [ExperimentSpec](#api-v1-beta1-ExperimentSpec)
59+
60+
<a name="api-v1-beta1-ExperimentSpec"></a>
61+
62+
### ExperimentSpec
63+
Specification of an Experiment. Experiment represents a single optimization run over a feasible space.
64+
Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.
65+
It is assumed that objective function f(x) does not change in the course of an Experiment.
66+
67+
68+
| Field | Type | Label | Description |
69+
| ----- | ---- | ----- | ----------- |
70+
| parameter_specs | [ExperimentSpec.ParameterSpecs](#api-v1-beta1-ExperimentSpec-ParameterSpecs) | | |
71+
| distribution | [ExperimentSpec.Distribution](#api-v1-beta1-ExperimentSpec-Distribution) | | |
72+
| objective | [ObjectiveSpec](#api-v1-beta1-ObjectiveSpec) | | Objective specification for the Experiment. |
73+
| algorithm | [AlgorithmSpec](#api-v1-beta1-AlgorithmSpec) | | HP or NAS algorithm specification for the Experiment. |
74+
| early_stopping | [EarlyStoppingSpec](#api-v1-beta1-EarlyStoppingSpec) | | Early stopping specification for the Experiment. |
75+
| parallel_trial_count | [int32](#int32) | | How many Trials can be processed in parallel. |
76+
| max_trial_count | [int32](#int32) | | Max completed Trials to mark Experiment as succeeded. |
77+
| nas_config | [NasConfig](#api-v1-beta1-NasConfig) | | NAS configuration for the Experiment. |
78+
79+
<a name="api-v1-beta1-ExperimentSpec-Distribution"></a>
80+
81+
### ExperimentSpec.Distribution
82+
List of Distribution.
83+
84+
85+
| Field | Type | Label | Description |
86+
| ----- | ---- | ----- | ----------- |
87+
| distributions | [Distribution](#api-v1-beta1-Distribution) | repeated | |
88+
89+
<a name="api-v1-beta1-Distribution"></a>
90+
91+
### Distribution
92+
Config for a hyperparameter.
93+
Katib will create each Hyper parameter from this config.
94+
95+
96+
| Field | Type | Label | Description |
97+
| ----- | ---- | ----- | ----------- |
98+
| name | [string](#string) | | Name of the Distribution. |
99+
| distribution_type | [DistributionType](#api-v1-beta1-DistributionType) | | Type of the Distribution. |
100+
| feasible_space | [FeasibleSpace](#api-v1-beta1-FeasibleSpace) | | FeasibleSpace for the Distribution. |
101+
102+
<a name="api-v1-beta1-DistributionType"></a>
103+
104+
### DistributionType
105+
Types of value for HyperParameter Distributions.
106+
107+
| Name | Number | Description |
108+
| ---- | ------ | ----------- |
109+
| UNIFORM | 0 | Continuous uniform distribution. Samples values evenly between a minimum and maximum value. Use &#34;Max/Min&#34;. Use &#34;Step&#34; for `q`. |
110+
| LOGUNIFORM | 1 | Samples values such that their logarithm is uniformly distributed. Use &#34;Max/Min&#34;. Use &#34;Step&#34; for `q`. |
111+
| NORMAL | 2 | Normal (Gaussian) distribution type. Samples values according to a normal distribution characterized by a mean and standard deviation. Use &#34;Max/Min&#34;. Use &#34;Step&#34; for `q`. |
112+
| LOGNORMAL | 3 | Log-normal distribution type. Samples values such that their logarithm is normally distributed. Use &#34;Max/Min&#34;. Use &#34;Step&#34; for `q`. |
113+
| CATEGORICAL | 4 | Categorical type. Use &#34;List&#34; as string. |
114+
115+
116+
const (
117+
CategoricalDistribution Distribution = "categorical"
118+
UniformDistribution Distribution = "uniform"
119+
LogUniformDistribution Distribution = "logUniform"
120+
NormalDistribution Distribution = "normal"
121+
LogNormalDistribution Distribution = "logNormal"
122+
)
123+
124+
<a name="api-v1-beta1-FeasibleSpace"></a>
125+
126+
### FeasibleSpace
127+
Feasible space for optimization.
128+
Int and Double type use Max/Min.
129+
Discrete and Categorical type use List.
130+
131+
132+
| Field | Type | Label | Description |
133+
| ----- | ---- | ----- | ----------- |
134+
| max | [string](#string) | | Max Value |
135+
| min | [string](#string) | | Minimum Value |
136+
| list | [string](#string) | repeated | List of Values. |
137+
| step | [string](#string) | | Step for double or int parameter or q for quantization|
138+
139+
140+
55141
## Experiment API changes
56142
Scope: `pkg/apis/controller/experiments/v1beta1/experiment_types.go`
57143
- Adding new field `Distribution` to `ParameterSpec`

0 commit comments

Comments
 (0)