TrialTemplates should be per-namespace and not always live in kubeflow namespace

[kimwnasptd]

/kind feature

Problem statement
Right now all the TrialTemplates reside in the kubeflow namespace. This has the following drawbacks:

1. Every team or user will need to be using the same templates
2. We will need to give permissions to all users for interacting with ConfigMaps in the kubeflow namespace, to allow them to edit the TrialTemplates

Describe the solution you'd like
Instead of deploying the TrialTemplates only in the kubeflow namespace we can expect them to exist in each namespace.

This will allow users to be able to configure and interact only with the Templates in their namespace. We can also enforce authorization checks with SubjectAccessReviews, since the we will have per-namespace resources.

Anything else you would like to add:
An important part to also discuss for this UX is how to initially create the ConfigMaps in each namespace, if they don't exist.

One solution we've been discussing with @andreyvelich would be to have the web app detect that no ConfigMaps are found in the current namespace and ask the user if they'd like the app to create some default ConfigMaps.

WDYT @andreyvelich @johnugeorge @gaocegege ?

[johnugeorge]

The configmaps in Kubeflow that we are talking about, are just examples right? does it make sense to create examples in every namespace?

[andreyvelich]

@johnugeorge I think it makes sense to create Trial template ConfigMap in user's namespace if user only wants it.

One solution we've been discussing with @andreyvelich would be to have the web app detect that no ConfigMaps are found in the current namespace and ask the user if they'd like the app to create some default ConfigMaps.

This sounds good.

How we want to deal with users who have access to all namespaces ? Do we still want to have Trial templates in Kubeflow namespace as an example ?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreyvelich]

/lifecycle frozen