Skip to content

Commit 46a5be1

Browse files
committed
Address Review Comments
1 parent 22e4467 commit 46a5be1

File tree

1 file changed

+17
-10
lines changed

1 file changed

+17
-10
lines changed

docs/proposals/conformance-test.md

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,22 @@ Conformance program for AutoML and Training WG should follow the same goals as P
2828
- The tests should cover basic functionality of Katib and the Training Operator.
2929
It will not cover all features.
3030
- The tests are expected to evolve in the future versions.
31+
- The tests should have a well documented and short list of set-up requirements.
32+
- The tests should install and complete in a relatively short period of time
33+
with suggested minimum infrastructure requirements
34+
(e.g. 3 nodes, 24 vCPU, 64 GB RAM, 500 GB Disk).
3135

3236
## Kubeflow Conformance
3337

34-
Kubeflow conformance consists the 3 category of tests:
38+
Initially the Kubeflow conformance will include the CRD based tests.
39+
In the future, API and UI based tests may be added. Kubeflow conformance consists
40+
the 3 category of tests:
41+
42+
- CRD-based tests
43+
44+
Most of Katib and Training Operator functionality are based on Kubernetes CRD.
45+
46+
**This document will define a design for CRD-based tests for Katib and the Training Operator.**
3547

3648
- API-based tests
3749

@@ -42,12 +54,6 @@ Kubeflow conformance consists the 3 category of tests:
4254
In the following versions, we should design conformance program for the
4355
Katib API-based tests.
4456

45-
- CRD-based tests
46-
47-
Most of Katib and Training Operator functionality are based on Kubernetes CRD.
48-
49-
**This document will define a design for CRD-based tests for Katib and the Training Operator.**
50-
5157
- UI-based tests
5258

5359
In the following versions, we should design conformance program for the
@@ -73,7 +79,7 @@ All tests will be run in the _kf-conformance_ namespace inside the separate cont
7379
That will help to avoid environment variance and improve fault tolerance. Driver is required to trigger the deployment and download the results.
7480

7581
- We are going to use
76-
[the unify Makefile](https://github.com/kubeflow/kubeflow/blob/2fa0d3665234125aeb8cebe8fe44f0a5a50791c5/conformance/1.5/Makefile)
82+
[the unified Makefile](https://github.com/kubeflow/kubeflow/blob/2fa0d3665234125aeb8cebe8fe44f0a5a50791c5/conformance/1.5/Makefile)
7783
for all Kubeflow conformance tests. Distributions (_driver_ on the diagram)
7884
need to run the following Makefile commands:
7985

@@ -135,6 +141,7 @@ That will help to avoid environment variance and improve fault tolerance. Driver
135141
- The above report can be downloaded from the test deployment by running `make report`.
136142

137143
- When all reports have been collected, the distributions are going to create PR
138-
to publish the reports. The Kubeflow Conformance Committee will verify it and
139-
make the distribution
144+
to publish the reports and to update the appropriate [Kubeflow Documentation](https://www.kubeflow.org/)
145+
on conformant Kubeflow distributions. The Kubeflow Conformance Committee will
146+
verify it and make the distribution
140147
[Certified Kubeflow](https://github.com/kubeflow/community/blob/master/proposals/kubeflow-conformance-program-proposal.md#overview).

0 commit comments

Comments
 (0)