@@ -28,10 +28,22 @@ Conformance program for AutoML and Training WG should follow the same goals as P
28
28
- The tests should cover basic functionality of Katib and the Training Operator.
29
29
It will not cover all features.
30
30
- The tests are expected to evolve in the future versions.
31
+ - The tests should have a well documented and short list of set-up requirements.
32
+ - The tests should install and complete in a relatively short period of time
33
+ with suggested minimum infrastructure requirements
34
+ (e.g. 3 nodes, 24 vCPU, 64 GB RAM, 500 GB Disk).
31
35
32
36
## Kubeflow Conformance
33
37
34
- Kubeflow conformance consists the 3 category of tests:
38
+ Initially the Kubeflow conformance will include the CRD based tests.
39
+ In the future, API and UI based tests may be added. Kubeflow conformance consists
40
+ the 3 category of tests:
41
+
42
+ - CRD-based tests
43
+
44
+ Most of Katib and Training Operator functionality are based on Kubernetes CRD.
45
+
46
+ ** This document will define a design for CRD-based tests for Katib and the Training Operator.**
35
47
36
48
- API-based tests
37
49
@@ -42,12 +54,6 @@ Kubeflow conformance consists the 3 category of tests:
42
54
In the following versions, we should design conformance program for the
43
55
Katib API-based tests.
44
56
45
- - CRD-based tests
46
-
47
- Most of Katib and Training Operator functionality are based on Kubernetes CRD.
48
-
49
- ** This document will define a design for CRD-based tests for Katib and the Training Operator.**
50
-
51
57
- UI-based tests
52
58
53
59
In the following versions, we should design conformance program for the
@@ -73,7 +79,7 @@ All tests will be run in the _kf-conformance_ namespace inside the separate cont
73
79
That will help to avoid environment variance and improve fault tolerance. Driver is required to trigger the deployment and download the results.
74
80
75
81
- We are going to use
76
- [ the unify Makefile] ( https://github.com/kubeflow/kubeflow/blob/2fa0d3665234125aeb8cebe8fe44f0a5a50791c5/conformance/1.5/Makefile )
82
+ [ the unified Makefile] ( https://github.com/kubeflow/kubeflow/blob/2fa0d3665234125aeb8cebe8fe44f0a5a50791c5/conformance/1.5/Makefile )
77
83
for all Kubeflow conformance tests. Distributions (_ driver_ on the diagram)
78
84
need to run the following Makefile commands:
79
85
@@ -135,6 +141,7 @@ That will help to avoid environment variance and improve fault tolerance. Driver
135
141
- The above report can be downloaded from the test deployment by running ` make report ` .
136
142
137
143
- When all reports have been collected, the distributions are going to create PR
138
- to publish the reports. The Kubeflow Conformance Committee will verify it and
139
- make the distribution
144
+ to publish the reports and to update the appropriate [ Kubeflow Documentation] ( https://www.kubeflow.org/ )
145
+ on conformant Kubeflow distributions. The Kubeflow Conformance Committee will
146
+ verify it and make the distribution
140
147
[ Certified Kubeflow] ( https://github.com/kubeflow/community/blob/master/proposals/kubeflow-conformance-program-proposal.md#overview ) .
0 commit comments