Keras 2.4 release adds dependency on tensorflow>=2.2.0, which breaks earlier installs

[cliffwoolley]

Yesterday, June 17, Keras 2.4.0 was released. This is the first numbered release since last October (which was 2.3.1).

Keras 2.4 now brings in tensorflow>=2.2 as a dependency (starting from #14121 , which was merged yesterday). It looks like this is because Keras 2.4 basically now redirects to tf.keras.

The problem, however, is one of compatibility: any user workflow that had pip install keras or packages that have keras in their requirements.txt/setup.py (without pinning to 'keras<2.4' or to some specific earlier version, at least) will now end up replacing existing TensorFlow 1.15 or TensorFlow 2.1 installations (which are the LTS versions as far as I'm aware) with TF 2.2.

I'm not quite sure what to suggest should be done as a corrective action at this point; even if this had been numbered as for example, "Keras 3.0" then still packages that don't pin to a particular maximum Keras version would have fallen into this trap.

Already some NVIDIA customers have started hitting this, so I wanted to open the discussion, at least.

[cliffwoolley]

/cc @fchollet

[tgaddair]

Thanks for raising this @cliffwoolley. I can confirm this change is incompatible with tensorflow-gpu and tf-nightly, resulting in multiple overlapping installations of TensorFlow and errors like the following:

AttributeError: module 'tensorflow.python.keras.utils.generic_utils' has no attribute 'populate_dict_with_module_objects'

[fchollet]

Incompatibility with tf-nightly is a problem. I will remove the version requirement.

[DEKHTIARJonathan]

@fchollet unfortunately your fix won't work to address the issue of tf-nightly and tensorflow-gpu.
You still have that hard dependency on tensorflow: https://github.com/keras-team/keras/blob/master/setup.py#L22

What I would suggest is the following:

1. Remove completely the dependency on TF from setup.py

2. Add this script in __init__.py :

from distutils.version import LooseVersion
try:
   import tensorflow as tf
   if LooseVersion(tf.__version__) < LooseVersion("2.2.0"):
       raise RuntimeError("Keras requires a version of Tensorflow >= 2.2. Please upgrade")
except ImportError:
   raise ImportError("Tensorflow >= 2.2 is not installed. Please run `pip install tensorflow>=2.2`")

This will effectively make any installation of Keras to fail on import if the condition on Tensorflow does not a version condition.
This solution also allows to use different tensorflow metapackages.

[tgaddair]

To add on to @DEKHTIARJonathan's suggestion, I think it should be possible to dynamically add tensorflow>=2.2 to the install_requires list only if TensorFlow isn't already installed (so they can get the benefit of auto-installing TensorFlow, consistent with the original PR), so something like:

from distutils.version import LooseVersion

requires_list = ['numpy>=1.9.1',
                        'scipy>=0.14',
                        'pyyaml',
                        'h5py']

try:
   import tensorflow as tf
   if LooseVersion(tf.__version__) < LooseVersion("2.2.0"):
       raise RuntimeError("Keras requires a version of Tensorflow >= 2.2. Please upgrade")
except ImportError:
   requires_list += ['tensorflow>=2.2']

...
setup(install_requires=requires_list, ...)

[DEKHTIARJonathan]

@tgaddair so long as keras is shipped as an sdist that will work. Totally agree with you

[fchollet]

Updated to remove the TF requirement and instead run an ImportError at runtime.

[sb4267]

ImportError: Keras requires TensorFlow 2.2 or higher. Install TensorFlow via `pip install tensorflow'

getting this error after the latest release

[cliffwoolley]

@sb4267 Yes, that is intentional. If you need a TensorFlow version prior to 2.2, you should pin Keras to keras<2.4 .

[cliffwoolley]

7b76bed for Keras 2.4.2 effectively resolves this issue.

Explanation: Keras 2.4.0 installation would silently swap out existing TensorFlow installations with TF 2.2, which may be undesirable as discussed above and is why this issue was opened.

Starting from Keras 2.4.2, rather than automatically swapping TF versions when installing keras, the new behavior is that import keras will throw the runtime error quoted by @sb4267 above if the user has incorrectly mixed the new Keras with an earlier TensorFlow. The user can then choose either upgrading to tensorflow>=2.2 or downgrading to keras<2.4.

Closing.