Developer docs need to account for the native parser

[stroxler]

A python developer naively following our README will now run into a problem when they try to pip install -e .:

> pip install -e .
Obtaining file:///home/stroxler/kode/LibCST
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Requirement already satisfied: typing-inspect>=0.4.0 in /home/stroxler/.pyenv/versions/3.9.7/envs/LibCST-3.9.7/lib/python3.9/site-packages (from libcst==0.4.0a3.dev9+g1337022) (0.7.1)
Requirement already satisfied: typing-extensions>=3.7.4.2 in /home/stroxler/.pyenv/versions/3.9.7/envs/LibCST-3.9.7/lib/python3.9/site-packages (from libcst==0.4.0a3.dev9+g1337022) (4.0.1)
Requirement already satisfied: pyyaml>=5.2 in /home/stroxler/.pyenv/versions/3.9.7/envs/LibCST-3.9.7/lib/python3.9/site-packages (from libcst==0.4.0a3.dev9+g1337022) (6.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /home/stroxler/.pyenv/versions/3.9.7/envs/LibCST-3.9.7/lib/python3.9/site-packages (from typing-inspect>=0.4.0->libcst==0.4.0a3.dev9+g1337022) (0.4.3)
Installing collected packages: libcst
  Attempting uninstall: libcst
    Found existing installation: libcst 0.4.0a3.dev16+gf5d69ca.d20220107
    Uninstalling libcst-0.4.0a3.dev16+gf5d69ca.d20220107:
      Successfully uninstalled libcst-0.4.0a3.dev16+gf5d69ca.d20220107
  Running setup.py develop for libcst
    ERROR: Command errored out with exit status 1:
     command: /home/stroxler/.pyenv/versions/3.9.7/envs/LibCST-3.9.7/bin/python3.9 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/stroxler/kode/LibCST/setup.py'"'"'; __file__='"'"'/home/stroxler/kode/LibCST/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/stroxler/kode/LibCST/
    Complete output (9 lines):
    running develop
    /tmp/pip-build-env-x6so18fg/overlay/lib/python3.9/site-packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /tmp/pip-build-env-x6so18fg/overlay/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running egg_info
    writing manifest file 'libcst.egg-info/SOURCES.txt'
    running build_ext
    error: [Errno 2] No such file or directory: 'cargo'

I can see at least two paths forward:

• In the short term could allow disabling the native parser build based on an environment variable
• We eventually need to document how to set up rust, or at any rate point folks to the rustup page.

[stroxler]

My thoughts on this:

• We definitely should document rust setup.
  • It appears to me that all you really need is to install cargo (e.g. via rustup)
  • I have a docker example of seting up an environment from bare ubuntu here
  • It's unclear to me whether a cargo build is needed; pip seems to hang if I don't
    run cargo first but maybe I just got too impatient.
• There's a case to be made for also allowing a no-native-parser pip install:
  • obviously the rust setup is a slight bit of overhead for a dev only working on the python side
  • more importantly, the first time we do a rust build it takes quite a while to run

I don't have a strong opinion either way.

[martindemello]

it worked for me without a cargo build

[stroxler]

Closing this as the cargo dependency is now recorded with a link to rustup docs as one possible approach.

We still want docs on the rust code in general, and ideally to turn it into a crate, but the immediate problem is fixed.