pip 19.0 fails to install packages that import to-be-installed package from CWD

[jeanlaroche]

Environment

• pip version: 19.0
• Python version: 3.6
• OS: MacOS

Description
When running pip install pyinstaller==3.4 with pip 19.0 we are getting an install error. ModuleNotFoundError: No module named 'PyInstaller'

Expected behavior
Expect pyinstall to be installed, as it is with pip 18.1

How to Reproduce
Using python3:
pip install pyinstaller=3.4

Output

pip install pyinstaller==3.4
Collecting pip
  Using cached https://files.pythonhosted.org/packages/60/64/73b729587b6b0d13e690a7c3acd2231ee561e8dd28a58ae1b0409a5a2b20/pip-19.0-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
Successfully installed pip-19.0
(BuildVEnv) jlaroche-mbp:TrackSense$ pip install pyinstaller
Collecting pyinstaller
  Using cached https://files.pythonhosted.org/packages/03/32/0e0de593f129bf1d1e77eed562496d154ef4460fd5cecfd78612ef39a0cc/PyInstaller-3.4.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  Complete output from command /Users/jlaroche/Dev/uapkg/packages/system/algo/BuildVEnv/bin/python3 /Users/jlaroche/Dev/uapkg/packages/system/algo/BuildVEnv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/j6/7t8sg1vj4q97zhh9z5cdmxbm4rz935/T/tmps3z6flnv:
  Traceback (most recent call last):
    File "/Users/jlaroche/Dev/uapkg/packages/system/algo/BuildVEnv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
      main()
    File "/Users/jlaroche/Dev/uapkg/packages/system/algo/BuildVEnv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/jlaroche/Dev/uapkg/packages/system/algo/BuildVEnv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/j6/7t8sg1vj4q97zhh9z5cdmxbm4rz935/T/pip-build-env-lo_ir5_f/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 115, in get_requires_for_build_wheel
      return _get_build_requires(config_settings, requirements=['wheel'])
    File "/private/var/folders/j6/7t8sg1vj4q97zhh9z5cdmxbm4rz935/T/pip-build-env-lo_ir5_f/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 101, in _get_build_requires
      _run_setup()
    File "/private/var/folders/j6/7t8sg1vj4q97zhh9z5cdmxbm4rz935/T/pip-build-env-lo_ir5_f/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 85, in _run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 20, in <module>
      from PyInstaller import __version__ as version, HOMEPATH, PLATFORM
  ModuleNotFoundError: No module named 'PyInstaller'

---

Maintainer note on timeline: See #6163 (comment)

[pradyunsg]

This seems to be an issue with either how pyinstaller is importing itself for installation.

It might be a good idea to file an issue over at the PyInstaller folks.

[rt121212121]

We currently use 18.1, and upgrading to 19.0 causes this problem for us as well. There is a related issue on the PyInstaller repo, where it is because pip '' is not in sys.path.

pyinstaller/pyinstaller#2730

[lithammer]

I think this is a pretty common workflow. You put __version__ = "1.2.3" in foo/__init__.py and then do import foo in setup.py so that you don't have to specify the version in two places. And any user of the the library can inspect the version according to PEP 396.

# foo/__init__.py
__version__ = "1.2.3"

# setup.py
from setuptools import setup

import foo

setup(..., version=foo.__version__)

[lithammer]

Also, this only happens if you have a pyproject.toml file (and setup.py). Remove it and the installation works fine. So there seems to be some differences in behaviour there. Maybe the traditional way modifies sys.path/PYTHONPATH?

[lithammer]

Ah, I think I get what is happening. Since by using a pyproject.toml file, you're basically telling pip you want to use PEP 517/518.

# pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]

The above tells pip that it needs setuptools and wheel to build PyInstaller. But in the case of PyInstaller, it's also got this in its setup.py:

# setup.py
from PyInstaller import __version__

From a PEP 517 perspective, aside from setuptools and wheel, it means it needs itself to build. Which is of course a bit weird.

# pyproject.toml
[build-system]
requires = ["setuptools", "wheel", "PyInstaller"]

[dstufft]

As @cjerdonek mentioned in #6175 (comment), could someone please check if --no-use-pep517 fixes this for them?

I suspect the cause of this issue is that build isolation or the PEP 517 code isn't making sure that the root of the package directory is on the sys.path, because pandas has a versioneer.py sitting next to setup.py. I recall this coming up at some point, but I don't remember off the top of my head what that discussion was. This might be considered an issue with the setuptools build backend instead of pip, or it might be the fault of pip's isolation mechanism.

[lithammer]

[...] could someone please check if --no-use-pep517 fixes this for them?

PyInstaller installs fine with --no-use-pep517.

[dstufft]

Ok, then that's certainly an issue with the new PEP 517 code and I'm pretty sure the issue is just that the directory containing the project root hasn't been added to sys.path. Maybe @pfmoore will have a better sense of if that should be pip's responsbility or setuptools.

[ashb]

If it helps another example of this (via apache-airflow is pip install pendulum==1.4.4 fails, but pip install --no-use-pep517 pendulum==1.4.4 works.

The stack trace we get is similar:

Collecting pendulum==1.4.4
  Using cached https://files.pythonhosted.org/packages/85/a5/9fc15751f9725923b170ad37d6c61031fc9e941bafd5288ca6ee51233284/pendulum-1.4.4.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  Complete output from command /Users/ash/.virtualenvs/clean-airflow/bin/python3.7 /Users/ash/.virtualenvs/clean-airflow/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/lr/9jc9vkgn025fn6jmwm4mv4_w0000gn/T/tmprosed3kj:
  Traceback (most recent call last):
    File "/Users/ash/.virtualenvs/clean-airflow/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
      main()
    File "/Users/ash/.virtualenvs/clean-airflow/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/ash/.virtualenvs/clean-airflow/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/lr/9jc9vkgn025fn6jmwm4mv4_w0000gn/T/pip-build-env-g__m0jh6/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 115, in get_requires_for_build_wheel
      return _get_build_requires(config_settings, requirements=['wheel'])
    File "/private/var/folders/lr/9jc9vkgn025fn6jmwm4mv4_w0000gn/T/pip-build-env-g__m0jh6/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 101, in _get_build_requires
      _run_setup()
    File "/private/var/folders/lr/9jc9vkgn025fn6jmwm4mv4_w0000gn/T/pip-build-env-g__m0jh6/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 85, in _run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 47, in <module>
      from build import *
    File "/Users/ash/.virtualenvs/clean-airflow/lib/python3.7/site-packages/pip/_vendor/pep517/build.py", line 7, in <module>
      from pip._vendor import pytoml
  ModuleNotFoundError: No module named 'pip'