Dependencies too strict for numpy and scipy

[juliuskittler]

Issue

I have a request regarding the dependencies for mljar-supervised = "0.10.4". In particular, my question is whether it would be possible to allow for older versions of numpy and scipy (maybe in future releases).

I am currently trying to install mljar-supervised = "0.10.4" along with tensorflow = "^2.2". It seems to me that these two dependencies should definitely be compatible in one application. However, this is not possible due to the following reason: because mljar-supervised (0.10.4) depends on both numpy (>=1.20.0) and scipy (1.6.1), mljar-supervised (0.10.4) is incompatible with tensorflow (>=2.2,<3.0)

Full dependency issue

Because no versions of tensorflow match >2.2,<2.2.1 || >2.2.1,<2.2.2 || >2.2.2,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.5.0 || >2.5.0,<3.0
   and tensorflow (2.2.0) depends on scipy (1.4.1), tensorflow (>=2.2,<2.2.1 || >2.2.1,<2.2.2 || >2.2.2,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1).
  And because tensorflow (2.2.1) depends on numpy (>=1.16.0,<1.19.0)
   and tensorflow (2.2.2) depends on numpy (>=1.16.0,<1.19.0), tensorflow (>=2.2,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0).
  And because tensorflow (2.3.0) depends on scipy (1.4.1)
   and tensorflow (2.3.1) depends on numpy (>=1.16.0,<1.19.0), tensorflow (>=2.2,<2.3.2 || >2.3.2,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0).
  And because tensorflow (2.3.2) depends on numpy (>=1.16.0,<1.19.0)
   and tensorflow (2.4.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.2,<2.4.1 || >2.4.1,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0 || >=1.19.2,<1.20.0).
  And because tensorflow (2.4.1) depends on numpy (>=1.19.2,<1.20.0)
   and tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.2,<3.0) requires numpy (>=1.16.0,<1.19.0 || >=1.19.2,<1.20.0) or scipy (1.4.1).
  And because mljar-supervised (0.10.4) depends on both numpy (>=1.20.0) and scipy (1.6.1), mljar-supervised (0.10.4) is incompatible with tensorflow (>=2.2,<3.0).
  So, because property-prediction-challenge depends on both tensorflow (^2.2) and mljar-supervised (0.10.4), version solving failed.

Reproducible example

• Step 1: Install poetry dependency management package with pip install poetry==1.1.6
• Step 2: Create a new folder and put the following into a new file pyproject.toml.

[tool.poetry]
name = "Dependency issues."
version = "0.1.0"
description = "Investigate dependency issues."
authors = ["nobody"]

[tool.poetry.dependencies]
python = "3.8.x"
tensorflow = "^2.2"
mljar-supervised = "0.10.4"

• Step 3: cd into the new folder and run poetry install

[pplonski]

@juliuskittler we can try to make requirements softer. Would you be able to check this?

Last week I added version 0.10.6 - I was tweaking requirements to pass conda-forge checks on different OS (MacOS, Win and Linux). The latest version doesn't have so strict requirements, but maybe it will be possible to use lower versions for scipy and numpy.

[juliuskittler]

@pplonski yes, I'd be very happy to check it! Please let me know how - when you're ready. (I can't see any tag for 0.10.6 on Github.)

[pplonski]

@juliuskittler great! thank you! I didnt add a tag in the github for 0.10.6 - don't have time to create release on GitHub ... I will fix it soon

[juliuskittler]

@pplonski alright, then I will use the 0.10.4 version from Github, adjust the requirements.txt locally and see if mljar-supervised works well - when installed with the adjusted requirements.txt? I'm also pretty occupied, but I'll try to do it in the next couple of weeks.

[pplonski]

OK! Your approach should work. Remember to use virtual env.

[juliuskittler]

Just an intermediate update. The most recent commit 8c3f9d1 (with more flexible dependencies) still results in dependency issues (with tensorflow = "^2.2"). Hence, I will have to look into allowing older package versions next.

[tool.poetry]
name = "Dependency issues."
version = "0.1.0"
description = "Investigate dependency issues."
authors = ["nobody"]

[tool.poetry.dependencies]
python = "3.8.x"
tensorflow = "^2.2"
mljar-supervised = { git = "https://github.com/mljar/mljar-supervised/", rev = "8c3f9d1ed527dfcfdaef91cf82e2779c5832e294" }

 Because no versions of tensorflow match >2.2,<2.2.1 || >2.2.1,<2.2.2 || >2.2.2,<2.2.3 || >2.2.3,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.3.3 || >2.3.3,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0
   and tensorflow (2.2.0) depends on scipy (1.4.1), tensorflow (>=2.2,<2.2.1 || >2.2.1,<2.2.2 || >2.2.2,<2.2.3 || >2.2.3,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.3.3 || >2.3.3,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1).
  And because tensorflow (2.2.1) depends on numpy (>=1.16.0,<1.19.0), tensorflow (>=2.2,<2.2.2 || >2.2.2,<2.2.3 || >2.2.3,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.3.3 || >2.3.3,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0).
  And because tensorflow (2.2.2) depends on numpy (>=1.16.0,<1.19.0)
   and tensorflow (2.2.3) depends on numpy (>=1.16.0,<1.19.0), tensorflow (>=2.2,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.3.2 || >2.3.2,<2.3.3 || >2.3.3,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0).
  And because tensorflow (2.3.0) depends on scipy (1.4.1)
   and tensorflow (2.3.1) depends on numpy (>=1.16.0,<1.19.0), tensorflow (>=2.2,<2.3.2 || >2.3.2,<2.3.3 || >2.3.3,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0).
  And because tensorflow (2.3.2) depends on numpy (>=1.16.0,<1.19.0)
   and tensorflow (2.3.3) depends on numpy (>=1.16.0,<1.19.0), tensorflow (>=2.2,<2.4.0 || >2.4.0,<2.4.1 || >2.4.1,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0).
  And because tensorflow (2.4.0) depends on numpy (>=1.19.2,<1.20.0)
   and tensorflow (2.4.1) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.2,<2.4.2 || >2.4.2,<2.5.0 || >2.5.0,<3.0) requires scipy (1.4.1) or numpy (>=1.16.0,<1.19.0 || >=1.19.2,<1.20.0).
  And because tensorflow (2.4.2) depends on numpy (>=1.19.2,<1.20.0)
   and tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.2,<3.0) requires numpy (>=1.16.0,<1.19.0 || >=1.19.2,<1.20.0) or scipy (1.4.1).
  And because mljar-supervised (rev 8c3f9d1ed527dfcfdaef91cf82e2779c5832e294) depends on both numpy (>=1.20.0) and scipy (>=1.6.1), mljar-supervised (rev 8c3f9d1ed527dfcfdaef91cf82e2779c5832e294) is incompatible with tensorflow (>=2.2,<3.0).
  So, because dependency issues. depends on both tensorflow (^2.2) and mljar-supervised (rev 8c3f9d1ed527dfcfdaef91cf82e2779c5832e294), version solving failed.

[juliuskittler]

@pplonski if you change your numpy dependency from numpy>=1.20 to numpy>=1.19.5, my dependency issue would be solved. Would you do this for future commits on mljar-supervised? Or do you see any issues here? (I deliberately took the closest possible numpy version relative to your current version 1.20, which would solve my dependency problem).

[tool.poetry]
name = "Dependency issues."
version = "0.1.0"
description = "Investigate dependency issues."
authors = ["nobody"]

[tool.poetry.dependencies]
python = "3.8.x"
tensorflow = "^2.2"
mljar-supervised = { git = "https://github.com/juliuskittler/mljar-supervised/", rev = "474741f95c3451e7c507e89dc5e179daf977ee6f" }

Reference: juliuskittler@474741f

[pplonski]

@juliuskittler thank you for checking this! I set numpy>=1.19.5 as you suggested and all tests passed https://travis-ci.com/github/mljar/mljar-supervised/builds/230609761 - changes are in the master branch.

I will need to check if all works on kaggle and colab kernels and update the conda installer.

[juliuskittler]

@pplonski Awesome, well thank you for making the adjustments :)