Set suggestion resource requests/limits per run

[dandawg]

/kind feature

Describe the solution you'd like
When submitting a katib experiment, I would like to be able to set custom suggestion resource requests/limits values that override the default requests/limits set in the config. At present, the suggestion resources are set in the global config, which applies to all runs.

Anything else you would like to add:
We have many different use-cases for Katib tuning, and some use cases (for example, those with many trials/many params) require more suggestion resources. It would be nice to be able to tweak the suggestion resources on a per-run level, or on a per-namespace level--although I recognize the latter probably requires more work.

---

Love this feature? Give it a 👍 We prioritize the features with the most 👍

[tenzen-y]

@dandawg Thanks for creating this!
We can overwrite resource requests/limits to suggestion services as shown below:

katib/manifests/v1beta1/components/controller/katib-config.yaml

Lines 51 to 58 in 1c74d65

	"enas": {
	"image": "docker.io/kubeflowkatib/suggestion-enas:latest",
	"resources": {
	"limits": {
	"memory": "200Mi"
	}
	}
	},

[tenzen-y]

Or, does that mean you'd like to set resource requests/limits to suggestion services per Experiment?

[johnugeorge]

Per experiment. Also, there are requests to add more fields to be exposed for suggestion deployment. Ref: #1737

Or, does that mean you'd like to set resource requests/limits to suggestion services per Experiment?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tenzen-y]

/lifecycle frozen