Skip to content

Commit 983a867

Browse files
andreyvelichtenzen-yeliaskoromilas
authored
Refactor Examples folder structure (#1691)
* Refactor Katib Examples * Fix links * Use Kind image Use kubectl wait * Update examples/v1beta1/kind-cluster/README.md Co-authored-by: Yuki Iwai <[email protected]> * Update examples/v1beta1/kind-cluster/README.md Co-authored-by: Yuki Iwai <[email protected]> * Increase timeout * Update docs/images-location.md Co-authored-by: Elias Koromilas <[email protected]> * Update examples/v1beta1/README.md Co-authored-by: Elias Koromilas <[email protected]> * Remove json * Add example links to training containers * Fix link * Update links to training-operator * Rename Trial settings to template * Rename Trial training containers to Trial images * Move NAS examples to Trial images * Add NAS links to README * Change TARGET DIR * Update examples/v1beta1/trial-images/mxnet-mnist/Dockerfile Co-authored-by: Yuki Iwai <[email protected]> * Update examples/v1beta1/trial-images/pytorch-mnist/Dockerfile Co-authored-by: Yuki Iwai <[email protected]> Co-authored-by: Yuki Iwai <[email protected]> Co-authored-by: Elias Koromilas <[email protected]>
1 parent 2db65b2 commit 983a867

File tree

81 files changed

+926
-609
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+926
-609
lines changed

‎.github/workflows/test-charmed-katib.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ jobs:
7575
- name: Test Katib
7676
run: |
7777
set -eux
78-
kubectl apply -f examples/v1beta1/random-example.yaml
78+
kubectl apply -f examples/v1beta1/hp-tuning/random.yaml
7979
8080
- name: Get pod statuses
8181
run: kubectl get all -A

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,7 @@ katib-mysql-7894994f88-5d4s5 1/1 Running 0 36s
178178
katib-ui-5767cfccdc-pwg2x 1/1 Running 0 36s
179179
```
180180

181-
For the Katib Experiments check the [complete examples list](examples).
181+
For the Katib Experiments check the [complete examples list](./examples/v1beta1).
182182

183183
# Documentation
184184

‎docs/images-location.md‎

Lines changed: 362 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,362 @@
1+
# Katib Images Location
2+
3+
Here you can find the location for images that are used in Katib.
4+
5+
## Katib Components Images
6+
7+
The following table shows images for the
8+
[Katib components](https://www.kubeflow.org/docs/components/katib/hyperparameter/#katib-components).
9+
10+
<table>
11+
<tbody>
12+
<tr align="center">
13+
<td>
14+
<b>Image Name</b>
15+
</td>
16+
<td>
17+
<b>Description</b>
18+
</td>
19+
<td>
20+
<b>Location</b>
21+
</td>
22+
</tr>
23+
<tr align="center">
24+
<td>
25+
<code>docker.io/kubeflowkatib/katib-controller</code>
26+
</td>
27+
<td>
28+
Katib Controller
29+
</td>
30+
<td>
31+
<a href="https://github.com/kubeflow/katib/tree/master/cmd/katib-controller/v1beta1/Dockerfile">Dockerfile</a>
32+
</td>
33+
</tr>
34+
<tr align="center">
35+
<td>
36+
<code>docker.io/kubeflowkatib/katib-ui</code>
37+
</td>
38+
<td>
39+
Katib User Interface
40+
</td>
41+
<td>
42+
<a href="https://github.com/kubeflow/katib/tree/master/cmd/ui/v1beta1/Dockerfile">Dockerfile</a>
43+
</td>
44+
</tr>
45+
<tr align="center">
46+
<td>
47+
<code>docker.io/kubeflowkatib/katib-db-manager</code>
48+
</td>
49+
<td>
50+
Katib DB Manager
51+
</td>
52+
<td>
53+
<a href="https://github.com/kubeflow/katib/tree/master/cmd/db-manager/v1beta1/Dockerfile">Dockerfile</a>
54+
</td>
55+
</tr>
56+
<tr align="center">
57+
<td>
58+
<code>docker.io/mysql</code>
59+
</td>
60+
<td>
61+
Katib MySQL DB
62+
</td>
63+
<td>
64+
<a href="https://github.com/docker-library/mysql/blob/c506174eab8ae160f56483e8d72410f8f1e1470f/8.0/Dockerfile.debian">Dockerfile</a>
65+
</td>
66+
</tr>
67+
<tr align="center">
68+
<td>
69+
<code>docker.io/kubeflowkatib/katib-cert-generator</code>
70+
</td>
71+
<td>
72+
Katib Cert Generator
73+
</td>
74+
<td>
75+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/cert-generator/v1beta1/Dockerfile">Dockerfile</a>
76+
</td>
77+
</tr>
78+
</tbody>
79+
</table>
80+
81+
## Katib Metrics Collectors Images
82+
83+
The following table shows images for the
84+
[Katib Metrics Collectors](https://www.kubeflow.org/docs/components/katib/experiment/#metrics-collector).
85+
86+
<table>
87+
<tbody>
88+
<tr align="center">
89+
<td>
90+
<b>Image Name</b>
91+
</td>
92+
<td>
93+
<b>Description</b>
94+
</td>
95+
<td>
96+
<b>Location</b>
97+
</td>
98+
</tr>
99+
<tr align="center">
100+
<td>
101+
<code>docker.io/kubeflowkatib/file-metrics-collector</code>
102+
</td>
103+
<td>
104+
File Metrics Collector
105+
</td>
106+
<td>
107+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/metricscollector/v1beta1/file-metricscollector/Dockerfile">Dockerfile</a>
108+
</td>
109+
</tr>
110+
<tr align="center">
111+
<td>
112+
<code>docker.io/kubeflowkatib/tfevent-metrics-collector</code>
113+
</td>
114+
<td>
115+
Tensorflow Event Metrics Collector
116+
</td>
117+
<td>
118+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile">Dockerfile</a>
119+
</td>
120+
</tr>
121+
</tbody>
122+
</table>
123+
124+
## Katib Suggestions and Early Stopping Images
125+
126+
The following table shows images for the
127+
[Katib Suggestions](https://www.kubeflow.org/docs/components/katib/experiment/#search-algorithms-in-detail)
128+
and the [Katib Early Stopping algorithms](https://www.kubeflow.org/docs/components/katib/early-stopping/).
129+
130+
<table>
131+
<tbody>
132+
<tr align="center">
133+
<td>
134+
<b>Image Name</b>
135+
</td>
136+
<td>
137+
<b>Description</b>
138+
</td>
139+
<td>
140+
<b>Location</b>
141+
</td>
142+
</tr>
143+
<tr align="center">
144+
<td>
145+
<code>docker.io/kubeflowkatib/suggestion-hyperopt</code>
146+
</td>
147+
<td>
148+
<a href="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/hyperopt/hyperopt">Hyperopt</a> Suggestion
149+
</td>
150+
<td>
151+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/hyperopt/v1beta1/Dockerfile">Dockerfile</a>
152+
</td>
153+
</tr>
154+
<tr align="center">
155+
<td>
156+
<code>docker.io/kubeflowkatib/suggestion-chocolate</code>
157+
</td>
158+
<td>
159+
<a href="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/AIworx-Labs/chocolate">Chocolate</a> Suggestion
160+
</td>
161+
<td>
162+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/chocolate/v1beta1/Dockerfile">Dockerfile</a>
163+
</td>
164+
</tr>
165+
<tr align="center">
166+
<td>
167+
<code>docker.io/kubeflowkatib/suggestion-skopt</code>
168+
</td>
169+
<td>
170+
<a href="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/scikit-optimize/scikit-optimize">Skopt</a> Suggestion
171+
</td>
172+
<td>
173+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/skopt/v1beta1/Dockerfile">Dockerfile</a>
174+
</td>
175+
</tr>
176+
<tr align="center">
177+
<td>
178+
<code>docker.io/kubeflowkatib/suggestion-optuna</code>
179+
</td>
180+
<td>
181+
<a href="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/optuna/optuna">Optuna</a> Suggestion
182+
</td>
183+
<td>
184+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/optuna/v1beta1/Dockerfile">Dockerfile</a>
185+
</td>
186+
</tr>
187+
<tr align="center">
188+
<td>
189+
<code>docker.io/kubeflowkatib/suggestion-goptuna</code>
190+
</td>
191+
<td>
192+
<a href="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/c-bata/goptuna">Goptuna</a> Suggestion
193+
</td>
194+
<td>
195+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/goptuna/v1beta1/Dockerfile">Dockerfile</a>
196+
</td>
197+
</tr>
198+
<tr align="center">
199+
<td>
200+
<code>docker.io/kubeflowkatib/suggestion-hyperband</code>
201+
</td>
202+
<td>
203+
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#hyperband">Hyperband</a> Suggestion
204+
</td>
205+
<td>
206+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/hyperband/v1beta1/Dockerfile">Dockerfile</a>
207+
</td>
208+
</tr>
209+
<tr align="center">
210+
<td>
211+
<code>docker.io/kubeflowkatib/suggestion-enas</code>
212+
</td>
213+
<td>
214+
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#enas">ENAS</a> Suggestion
215+
</td>
216+
<td>
217+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/nas/enas/v1beta1/Dockerfile">Dockerfile</a>
218+
</td>
219+
</tr>
220+
<tr align="center">
221+
<td>
222+
<code>docker.io/kubeflowkatib/suggestion-darts</code>
223+
</td>
224+
<td>
225+
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#differentiable-architecture-search-darts">DARTS</a> Suggestion
226+
</td>
227+
<td>
228+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/nas/darts/v1beta1/Dockerfile">Dockerfile</a>
229+
</td>
230+
</tr>
231+
<tr align="center">
232+
<td>
233+
<code>docker.io/kubeflowkatib/earlystopping-medianstop</code>
234+
</td>
235+
<td>
236+
<a href="https://www.kubeflow.org/docs/components/katib/early-stopping/#median-stopping-rule">Median Stopping Rule</a>
237+
</td>
238+
<td>
239+
<a href="https://github.com/kubeflow/katib/blob/master/cmd/earlystopping/medianstop/v1beta1/Dockerfile">Dockerfile</a>
240+
</td>
241+
</tr>
242+
</tbody>
243+
</table>
244+
245+
## Training Containers Images
246+
247+
The following table shows images for training containers which are used in the
248+
[Katib Trials](https://www.kubeflow.org/docs/components/katib/experiment/#packaging-your-training-code-in-a-container-image).
249+
250+
<table>
251+
<tbody>
252+
<tr align="center">
253+
<td>
254+
<b>Image Name</b>
255+
</td>
256+
<td>
257+
<b>Description</b>
258+
</td>
259+
<td>
260+
<b>Location</b>
261+
</td>
262+
</tr>
263+
<tr align="center">
264+
<td>
265+
<code>docker.io/kubeflowkatib/mxnet-mnist</code>
266+
</td>
267+
<td>
268+
MXNet MNIST example with collecting metrics time
269+
</td>
270+
<td>
271+
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/mxnet-mnist/Dockerfile">Dockerfile</a>
272+
</td>
273+
</tr>
274+
<tr align="center">
275+
<td>
276+
<code>docker.io/kubeflowkatib/pytorch-mnist</code>
277+
</td>
278+
<td>
279+
PyTorch MNIST example with printing metrics to the file or StdOut
280+
</td>
281+
<td>
282+
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/pytorch-mnist/Dockerfile">Dockerfile</a>
283+
</td>
284+
</tr>
285+
<tr align="center">
286+
<td>
287+
<code>gcr.io/kubeflow-ci/tf-mnist-with-summaries</code>
288+
</td>
289+
<td>
290+
Tensorflow MNIST example with saving metrics in the summaries
291+
</td>
292+
<td>
293+
<a href="https://github.com/kubeflow/training-operator/blob/2712f5667ec78f17d22288630f8719f0c08990ba/examples/tensorflow/mnist_with_summaries/Dockerfile">Dockerfile</a>
294+
</td>
295+
</tr>
296+
<tr align="center">
297+
<td>
298+
<code>docker.io/kubeflowkatib/xgboost-lightgbm</code>
299+
</td>
300+
<td>
301+
Distributed LightGBM example for XGBoostJob
302+
</td>
303+
<td>
304+
<a href="https://github.com/kubeflow/xgboost-operator/blob/9c8c97d0125a8156f12b8ef5b93f99e709fb57ea/config/samples/lightgbm-dist/Dockerfile">Dockerfile</a>
305+
</td>
306+
</tr>
307+
<tr align="center">
308+
<td>
309+
<code>docker.io/kubeflow/mpi-horovod-mnist</code>
310+
</td>
311+
<td>
312+
Distributed Horovod example for MPIJob
313+
</td>
314+
<td>
315+
<a href="https://github.com/kubeflow/mpi-operator/blob/947d396a9caf70d3c94bf587d5e5da32b70f0f53/examples/horovod/Dockerfile.cpu">Dockerfile</a>
316+
</td>
317+
</tr>
318+
<tr align="center">
319+
<td>
320+
<code>docker.io/inaccel/jupyter:lab</code>
321+
</td>
322+
<td>
323+
FPGA XGBoost with parameter tuning
324+
</td>
325+
<td>
326+
<a href="https://github.com/inaccel/jupyter/blob/master/lab/Dockerfile">Dockerfile</a>
327+
</td>
328+
</tr>
329+
<tr align="center">
330+
<td>
331+
<code>docker.io/kubeflowkatib/enas-cnn-cifar10-gpu</code>
332+
</td>
333+
<td>
334+
Keras CIFAR-10 CNN example for ENAS with GPU support
335+
</td>
336+
<td>
337+
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/enas-cnn-cifar10/Dockerfile.gpu">Dockerfile</a>
338+
</td>
339+
</tr>
340+
<tr align="center">
341+
<td>
342+
<code>docker.io/kubeflowkatib/enas-cnn-cifar10-cpu</code>
343+
</td>
344+
<td>
345+
Keras CIFAR-10 CNN example for ENAS with CPU support
346+
</td>
347+
<td>
348+
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/enas-cnn-cifar10/Dockerfile.cpu">Dockerfile</a>
349+
</td>
350+
</tr>
351+
<tr align="center">
352+
<td>
353+
<code>docker.io/kubeflowkatib/darts-cnn-cifar10</code>
354+
</td>
355+
<td>
356+
PyTorch CIFAR-10 CNN example for DARTS
357+
</td>
358+
<td>
359+
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/darts-cnn-cifar10/Dockerfile">Dockerfile</a>
360+
</td>
361+
</tr>
362+
</table>

0 commit comments

Comments
 (0)