Skip to content

Katib: Add components guide #2962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 22, 2021
Merged

Katib: Add components guide #2962

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
babd14d
Katib: Add components guide
andreyvelich Sep 20, 2021
d87d449
Add links for CRDs
andreyvelich Sep 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 42 additions & 14 deletions content/en/docs/components/katib/hyperparameter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ For an overview of the concepts around Katib and hyperparameter tuning, check th

Let's set up and configure Katib on your Kubernetes cluster with Kubeflow.

### Prerequisites

This is the minimal requirements to install Katib:

- Kubernetes >= 1.17
- `kubectl` >= 1.21

<a id="katib-install"></a>

### Installing Katib
Expand All @@ -26,23 +33,16 @@ To install Katib as part of Kubeflow, follow the
[Kubeflow installation guide](/docs/started/getting-started/).

If you want to install Katib separately from Kubeflow, or to get a later version
of Katib, you can use various Katib installs. Run the following command to clone
Katib repository:

```shell
git clone https://github.com/kubeflow/katib
cd katib
```

You can use one of the following Katib installations.
of Katib, you can use one of the following Katib installs. To install the specific
Katib release (e.g. `v0.11.1`), modify `ref=master` to `ref=v0.11.1`.

1. **Katib Standalone Installation**

Run the following command to deploy Katib with the main components
(`katib-controller`, `katib-ui`, `katib-mysql`, `katib-db-manager`, and `katib-cert-generator`):

```shell
make deploy
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=master"
```

This installation doesn't require any additional setup on your Kubernetes cluster.
Expand All @@ -53,7 +53,7 @@ You can use one of the following Katib installations.
[Cert Manager](https://cert-manager.io/docs/installation/kubernetes/) requirement:

```shell
kustomize build manifests/v1beta1/installs/katib-cert-manager | kubectl apply -f -
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-cert-manager?ref=master"
```

This installation uses Cert Manager instead of `katib-cert-generator`
Expand All @@ -65,7 +65,7 @@ You can use one of the following Katib installations.
Run the following command to deploy Katib with custom Database (DB) backend:

```shell
kustomize build manifests/v1beta1/installs/katib-external-db | kubectl apply -f -
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-external-db?ref=master"
```

This installation allows to use custom MySQL DB instead of `katib-mysql`.
Expand All @@ -79,7 +79,7 @@ You can use one of the following Katib installations.
Run the following command to deploy Katib on [OpenShift](https://docs.openshift.com/) v4.4+:

```shell
kustomize build ./manifests/v1beta1/installs/katib-openshift | oc apply -f - -l type!=local
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-openshift?ref=master"
```

This installation uses OpenShift service controller instead of `katib-cert-generator`
Expand All @@ -98,7 +98,35 @@ deploy [PersistentVolume (PV)](https://kubernetes.io/docs/concepts/storage/persi
to bind [PVC](https://github.com/kubeflow/katib/blob/master/manifests/v1beta1/components/mysql/pvc.yaml)
for the Katib DB component.

<a id="katib-ui"></a>
### Katib components

Run the following command to verify that Katib components are running:

```shell
$ kubectl get pods -n kubeflow

NAME READY STATUS RESTARTS AGE
katib-cert-generator-79g7d 0/1 Completed 0 79s
katib-controller-566595bdd8-8w7sx 1/1 Running 0 82s
katib-db-manager-57cd769cdb-vt7zs 1/1 Running 0 82s
katib-mysql-7894994f88-djp7m 1/1 Running 0 81s
katib-ui-5767cfccdc-v9fcs 1/1 Running 0 80s
```

- `katib-controller` - the controller to manage Katib Kubernetes CRDs
([`Experiment`](/docs/components/katib/overview/#experiment),
[`Suggestion`](/docs/components/katib/overview/#suggestion),
[`Trial`](/docs/components/katib/overview/#trial))

- `katib-ui` - the Katib user interface.

- `katib-db-manager` - the GRPC API server to control Katib DB interface.

- `katib-mysql` - the `mysql` DB backend to store Katib experiments metrics.

- (Optional) `katib-cert-generator` - the certificate generator for Katib
standalone installation. Learn more about the cert generator in the
[developer guide](https://github.com/kubeflow/katib/blob/master/docs/developer-guide.md#katib-cert-generator)

## Accessing the Katib UI

Expand Down
9 changes: 9 additions & 0 deletions content/en/docs/components/katib/overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,9 @@ the main configurations:
- **Search algorithm**: The algorithm to use when searching for the optimal
hyperparameter values.

Katib experiment is defined as a
[Kubernetes CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) .

For details of how to define your experiment, follow the guide to [running an
experiment](/docs/components/katib/experiment/).

Expand All @@ -174,6 +177,9 @@ A _suggestion_ is a set of hyperparameter values that the hyperparameter
tuning process has proposed. Katib creates a trial to evaluate the suggested
set of values.

Katib suggestion is defined as a
[Kubernetes CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) .

### Trial

A _trial_ is one iteration of the hyperparameter tuning process. A trial
Expand All @@ -183,6 +189,9 @@ The list of parameter assignments corresponds to a suggestion.
Each experiment runs several trials. The experiment runs the trials until it
reaches either the objective or the configured maximum number of trials.

Katib trial is defined as a
[Kubernetes CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) .

### Worker job

The _worker job_ is the process that runs to evaluate a trial and calculate
Expand Down
1 change: 1 addition & 0 deletions content/en/docs/reference/images.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ weight = 10
| kubeflowkatib/suggestion-enas | <https://github.com/kubeflow/katib/blob/master/cmd/suggestion/nas/enas/v1beta1/Dockerfile> |
| kubeflowkatib/suggestion-darts | <https://github.com/kubeflow/katib/blob/master/cmd/suggestion/nas/darts/v1beta1/Dockerfile> |
| kubeflowkatib/suggestion-goptuna | <https://github.com/kubeflow/katib/blob/master/cmd/suggestion/goptuna/v1beta1/Dockerfile> |
| kubeflowkatib/suggestion-optuna | <https://github.com/kubeflow/katib/blob/master/cmd/suggestion/optuna/v1beta1/Dockerfile> |
| kubeflowkatib/earlystopping-medianstop | <https://github.com/kubeflow/katib/blob/master/cmd/earlystopping/medianstop/v1beta1/Dockerfile> |
| kubeflowkatib/file-metrics-collector | <https://github.com/kubeflow/katib/blob/master/cmd/metricscollector/v1beta1/file-metricscollector/Dockerfile> |
| kubeflowkatib/tfevent-metrics-collector | <https://github.com/kubeflow/katib/blob/master/cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile> |
Expand Down