katib python sdk incompatible pinned version

[ryansteakley]

/kind bug

What steps did you take and what happened:

pip install latest kserve and katib, recieve

ERROR: Cannot install -r /tmp/requirements.txt (line 16), -r /tmp/requirements.txt (line 18) and ray[serve]==2.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    kserve 0.10.0 depends on grpcio>=1.34.0
    kubeflow-katib 0.15.0 depends on grpcio==1.41.1
    ray[serve] 2.0.0 depends on grpcio<=1.43.0 and >=1.42.0; python_version >= "3.10"

is it possible to change the require grpcio version in katib?
What did you expect to happen:
Able to install both to use with kubeflow

Impacted by this bug? Give it a 👍 We prioritize the issues with the most 👍

[tenzen-y]

@ryansteakley Thank you for creating this issue!

Our (katib) grpcio version seems so old.

https://pypi.org/project/grpcio/#history

So, we should upgrade the grpcio version or use soft restriction (grpcio>=1.41.1) instead of hard restriction (grpcio==1.41.1).

@andreyvelich @anencore94 What do you think?

[andreyvelich]

Thank you for creating this @ryansteakley!

It's always very hard to sync dependencies across various services 😄

Historically, we were using grpcio==1.41.1 for all of our Python services.
For example, TFEvent Collector or Optuna Suggestion.

As long as, the required version will work for us, I am ok with soft restriction.

[andreyvelich]

@johnugeorge Please take a look for this issue. We might want to include this change in Katib 0.15.1.

Since it might take time for us time to make another release, users can install Katib SDK from GitHub source as follows (When we push the change):

pip install git+https://github.com/kubeflow/katib.git@d2d9cab1cadd807166b7e3631017e746ac76a8fe#subdirectory=sdk/python/v1beta1

[johnugeorge]

This is a good discussion point. Similar to how we take care of controller dependencies across various kubeflow components, It would be good to do the same for sdk as well(basic installation) across Kubeflow components.