This issue serves as Katib sub-issue of the main issue: https://github.com/kubeflow/testing/issues/993. TLDR: Katib WG should start to research and choose a preferable testing solution for the E2E testing use case before June 2022. /cc @kubeflow/wg-automl-leads