[Installation]: Failed to build vLLM from source

[congcongchen123]

Your current environment

The output of `python collect_env.py`

Traceback (most recent call last):
File "/users/congcongchen/code/vllm/vllm/collect_env.py", line 17, in
from vllm.envs import environment_variables
ModuleNotFoundError: No module named 'vllm'

How you are installing vllm

conda create -n vllm python=3.12 -y
conda activate vllm
git clone https://github.com/vllm-project/vllm.git
cd vllm
conda install -y ccache
pip install --user -e . -vvv # This may take 5-10 minutes.

Full log

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.3/37.3 MB 222.3 MB/s eta 0:00:00
Downloading link https://files.pythonhosted.org/packages/0b/1f/03f52c282437a168ee2c7c14a1a0d0781a9a4a8962d84ac05c06b4c5b555/scipy-1.15.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.org/simple/scipy/) (requires-python:>=3.10) to /tmp/pip-unpack-wjt1pvqh/scipy-1.15.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Building wheels for collected packages: vllm
  Created temporary directory: /tmp/pip-wheel-4yvtdwd5
  Destination directory: /tmp/pip-wheel-4yvtdwd5
  Running command Building editable for vllm (pyproject.toml)
  /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/_subclasses/functional_tensor.py:276: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:81.)
    cpu = _conversion_method_template(device=torch.device("cpu"))
  running editable_wheel
  creating /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info
  writing /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/dependency_links.txt
  writing entry points to /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/entry_points.txt
  writing requirements to /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/requires.txt
  writing top-level names to /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/top_level.txt
  writing manifest file '/tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm.egg-info/SOURCES.txt'
  creating '/tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm-0.9.1.dev4+g9609327fa.cu118.dist-info'
  creating /tmp/pip-wheel-4yvtdwd5/.tmp-xcx_yw6y/vllm-0.9.1.dev4+g9609327fa.cu118.dist-info/WHEEL
  running build_py
  running build_ext
  Using MAX_JOBS=8192 as the number of jobs.
  Using NVCC_THREADS=128 as the number of nvcc threads.
  -- The CXX compiler identification is GNU 9.4.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Build type: RelWithDebInfo
  -- Target device: cuda
  -- Found Python: /home/aiscuser/.conda/envs/vllm/bin/python3.12 (found version "3.12.9") found components: Interpreter Development.Module Development.SABIModule
  -- Found python matching: /home/aiscuser/.conda/envs/vllm/bin/python3.12.
  -- Found CUDA: /usr/local/cuda (found version "11.8")
  -- The CUDA compiler identification is NVIDIA 11.8.89 with host compiler GNU 9.4.0
  -- Detecting CUDA compiler ABI info
  -- Detecting CUDA compiler ABI info - done
  -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
  -- Detecting CUDA compile features
  -- Detecting CUDA compile features - done
  -- Found CUDAToolkit: /usr/local/cuda/include (found version "11.8.89")
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- PyTorch: CUDA detected: 11.8
  -- PyTorch: CUDA nvcc is: /usr/local/cuda/bin/nvcc
  -- PyTorch: CUDA toolkit directory: /usr/local/cuda
  -- PyTorch: Header version is: 11.8
  -- Found Python: /home/aiscuser/.conda/envs/vllm/bin/python3.12 (found version "3.12.9") found components: Interpreter
  CMake Warning at /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:140 (message):
    Failed to compute shorthash for libnvrtc.so
  Call Stack (most recent call first):
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
    CMakeLists.txt:80 (find_package)


  CMake Warning (dev) at /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/cmake/data/share/cmake-4.0/Modules/FindPackageHandleStandardArgs.cmake:430 (message):
    The package name passed to find_package_handle_standard_args() (nvtx3) does
    not match the name of the calling package (Caffe2).  This can lead to
    problems in calling code that expects find_package() result variables
    (e.g., `_FOUND`) to follow a certain pattern.
  Call Stack (most recent call first):
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:178 (find_package_handle_standard_args)
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
    CMakeLists.txt:80 (find_package)
  This warning is for project developers.  Use -Wno-dev to suppress it.

  -- Could NOT find nvtx3 (missing: nvtx3_dir)
  CMake Warning at /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:184 (message):
    Cannot find NVTX3, find old NVTX instead
  Call Stack (most recent call first):
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
    CMakeLists.txt:80 (find_package)


  -- USE_CUDNN is set to 0. Compiling without cuDNN support
  -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
  -- USE_CUDSS is set to 0. Compiling without cuDSS support
  -- USE_CUFILE is set to 0. Compiling without cuFile support
  -- Autodetected CUDA architecture(s):  8.0 8.0 8.0 8.0 8.0 8.0 8.0 8.0
  -- Added CUDA NVCC flags for: -gencode;arch=compute_80,code=sm_80
  CMake Warning at /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
    static library kineto_LIBRARY-NOTFOUND not found.
  Call Stack (most recent call first):
    /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:125 (append_torchlib_if_found)
    CMakeLists.txt:80 (find_package)


  -- Found Torch: /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/torch/lib/libtorch.so
  -- CUDA target architectures: 8.0
  -- CUDA supported target architectures: 8.0
  -- FetchContent base directory: /scratch/AzureNfsServer_INPUT3/scratch/turing_westus3_nfs/users/congcongchen/code/vllm_public/vllm/.deps
  -- Enabling cumem allocator extension.
  -- CMake Version: 4.0.2
  -- CUTLASS 3.9.2
  -- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "11.8.89")
  -- CUDART: /usr/local/cuda/lib64/libcudart.so
  -- CUDA Driver: /usr/local/cuda/lib64/stubs/libcuda.so
  -- NVRTC: /usr/local/cuda/lib64/libnvrtc.so
  -- Default Install Location: install
  -- Found Python3: /home/aiscuser/.conda/envs/vllm/bin/python3.12 (found suitable version "3.12.9", minimum required is "3.5") found components: Interpreter
  -- CUDA Compilation Architectures: 70;72;75;80;86;87;89;90
  -- Enable caching of reference results in conv unit tests
  -- Enable rigorous conv problem sizes in conv unit tests
  -- Grid Dependency Control (GDC) is enabled for SM100 kernels (required for programmatic dependent launches).
  -- Using the following NVCC flags:
    --expt-relaxed-constexpr
    -ftemplate-backtrace-limit=0
    -DCUTLASS_TEST_LEVEL=0
    -DCUTLASS_TEST_ENABLE_CACHED_RESULTS=1
    -DCUTLASS_CONV_UNIT_TEST_RIGOROUS_SIZE_ENABLED=1
    -DCUTLASS_DEBUG_TRACE_LEVEL=0
    -Xcompiler=-Wconversion
    -Xcompiler=-fno-strict-aliasing
    -lineinfo
  -- Configuring cublas ...
  -- cuBLAS Disabled.
  -- Configuring cuBLAS ... done.
  -- Marlin generation script hash: 0fcae1cd7de8a1a79f7344bb65cf9436
  -- Last run Marlin generate script hash:
  CMake Error at CMakeLists.txt:332 (message):
    Marlin generation failed.  Result: "1"

    Check the log for details:
    /tmp/tmpd09duydv.build-temp/marlin_generation.log


  -- Configuring incomplete, errors occurred!
  Traceback (most recent call last):
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 139, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 340, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 263, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 290, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "<string>", line 270, in run
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
      self.build_extensions()
    File "<string>", line 229, in build_extensions
    File "<string>", line 207, in configure
    File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/subprocess.py", line 415, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/scratch/AzureNfsServer_INPUT3/scratch/turing_westus3_nfs/users/congcongchen/code/vllm_public/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DVLLM_TARGET_DEVICE=cuda', '-DCMAKE_C_COMPILER_LAUNCHER=ccache', '-DCMAKE_CXX_COMPILER_LAUNCHER=ccache', '-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache', '-DCMAKE_HIP_COMPILER_LAUNCHER=ccache', '-DVLLM_PYTHON_EXECUTABLE=/home/aiscuser/.conda/envs/vllm/bin/python3.12', '-DVLLM_PYTHON_PATH=/tmp/pip-build-env-0fmd0nj7/site:/home/aiscuser/.conda/envs/vllm/lib/python312.zip:/home/aiscuser/.conda/envs/vllm/lib/python3.12:/home/aiscuser/.conda/envs/vllm/lib/python3.12/lib-dynload:/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages:/tmp/pip-build-env-0fmd0nj7/normal/lib/python3.12/site-packages:/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_vendor', '-DFETCHCONTENT_BASE_DIR=/scratch/AzureNfsServer_INPUT3/scratch/turing_westus3_nfs/users/congcongchen/code/vllm_public/vllm/.deps', '-DNVCC_THREADS=128', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=64', '-DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc']' returned non-zero exit status 1.
  /tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py:1021: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `vllm` in editable mode.

          The following steps are recommended to help debug this problem:

          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.

          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:

              https://github.com/pypa/setuptools/issues

          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************

  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
      main()
    File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
      json_out["return_val"] = hook(**hook_input["kwargs"])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 303, in build_editable
      return hook(wheel_directory, config_settings, metadata_directory)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 468, in build_editable
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 317, in run_setup
      exec(code, locals())
    File "<string>", line 688, in <module>
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 186, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 139, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 340, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 263, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 290, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "<string>", line 270, in run
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
      self.build_extensions()
    File "<string>", line 229, in build_extensions
    File "<string>", line 207, in configure
    File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/subprocess.py", line 415, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/scratch/AzureNfsServer_INPUT3/scratch/turing_westus3_nfs/users/congcongchen/code/vllm_public/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DVLLM_TARGET_DEVICE=cuda', '-DCMAKE_C_COMPILER_LAUNCHER=ccache', '-DCMAKE_CXX_COMPILER_LAUNCHER=ccache', '-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache', '-DCMAKE_HIP_COMPILER_LAUNCHER=ccache', '-DVLLM_PYTHON_EXECUTABLE=/home/aiscuser/.conda/envs/vllm/bin/python3.12', '-DVLLM_PYTHON_PATH=/tmp/pip-build-env-0fmd0nj7/site:/home/aiscuser/.conda/envs/vllm/lib/python312.zip:/home/aiscuser/.conda/envs/vllm/lib/python3.12:/home/aiscuser/.conda/envs/vllm/lib/python3.12/lib-dynload:/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages:/tmp/pip-build-env-0fmd0nj7/normal/lib/python3.12/site-packages:/tmp/pip-build-env-0fmd0nj7/overlay/lib/python3.12/site-packages/setuptools/_vendor', '-DFETCHCONTENT_BASE_DIR=/scratch/AzureNfsServer_INPUT3/scratch/turing_westus3_nfs/users/congcongchen/code/vllm_public/vllm/.deps', '-DNVCC_THREADS=128', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=64', '-DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Building editable for vllm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/aiscuser/.conda/envs/vllm/bin/python3.12 /home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_editable /tmp/tmph3p2umqy
  cwd: /scratch/AzureNfsServer_INPUT3/scratch/turing_westus3_nfs/users/congcongchen/code/vllm_public/vllm
  Building editable for vllm (pyproject.toml) ... error
  ERROR: Failed building editable for vllm
Failed to build vllm
1 location(s) to search for versions of pip:
* https://pypi.org/simple/pip/
Fetching project page and analyzing links: https://pypi.org/simple/pip/
Getting page https://pypi.org/simple/pip/
Found index url https://pypi.org/simple/
Looking up "https://pypi.org/simple/pip/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/pip/ HTTP/1.1" 304 0
Fetched page https://pypi.org/simple/pip/ as application/vnd.pypi.simple.v1+json
 ...

Found link https://files.pythonhosted.org/packages/59/de/241caa0ca606f2ec5fe0c1f4261b0465df78d786a38da693864a116c37f4/pip-25.1.1.tar.gz (from https://pypi.org/simple/pip/) (requires-python:>=3.9), version: 25.1.1
Skipping link: not a file: https://pypi.org/simple/pip/
Given no hashes to check 249 links for project 'pip': discarding no candidates
Remote version of pip: 25.1.1
Local version of pip:  25.1
Was pip installed by pip? False
ERROR: Failed to build installable wheels for some pyproject.toml based projects (vllm)
Exception information:
Traceback (most recent call last):
  File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
             ^^^^^^^^^^^^
  File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 68, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aiscuser/.conda/envs/vllm/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 436, in run
    raise InstallationError(
pip._internal.exceptions.InstallationError: Failed to build installable wheels for some pyproject.toml based projects (vllm)
Removed build tracker: '/tmp/pip-build-tracker-1926im3g'

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[congcongchen123]

Hi @simon-mo , @mgoin I could not build vLLM from source with the latest main branch, however I was able to build successfully from older commit hash:

commit 051da7efe39591f256dea30c286ed7c920c0f7d2
Author: Lu Fang <30275821+houseroad@users.noreply.github.com>
Date:   Tue Mar 25 00:36:45 2025 -0700

    Fix CUDA kernel index data type in vllm/csrc/quantization/gptq_marlin/awq_marlin_repack.cu +10 (#15160)

    Signed-off-by: Lu Fang <lufang@fb.com>
    Co-authored-by: Richard Barnes <rbarnes@meta.com>

And from the error logs, it looks like related to Marlin, see

  CMake Error at CMakeLists.txt:332 (message):
    Marlin generation failed.  Result: "1"

    Check the log for details:
    /tmp/tmpd09duydv.build-temp/marlin_generation.log

I tried to open /tmp/tmpd09duydv.build-temp/marlin_generation.log, it doesn't exist.

Could anyone please help with this? Thanks a lot!

[simon-mo]

should be fixed by #17679

[aatkinson]

I got the same issue with latest vLLM

 -- Configuring cublas ...
  -- cuBLAS Disabled.
  -- Configuring cuBLAS ... done.
  -- Marlin generation script hash: 0fcae1cd7de8a1a79f7344bb65cf9436
  -- Last run Marlin generate script hash:
  CMake Error at CMakeLists.txt:332 (message):
    Marlin generation failed.  Result: "1"

    Check the log for details:
    /tmp/tmpn6lg_246.build-temp/marlin_generation.log


  -- Configuring incomplete, errors occurred!
  Traceback (most recent call last):
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 139, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 340, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 263, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/command/editable_wheel.py", line 290, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "<string>", line 270, in run
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
      self.build_extensions()
    File "<string>", line 229, in build_extensions
    File "<string>", line 207, in configure
    File "/home/aiscuser/miniconda3/envs/myenveditablevllm085/lib/python3.12/subprocess.py", line 415, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/scratch/AzureNfsServer_INPUT3/data/users/adatkins/dev/phivnext/yoco/yocov2/vllm085_main', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DVLLM_TARGET_DEVICE=cuda', '-DCMAKE_C_COMPILER_LAUNCHER=ccache', '-DCMAKE_CXX_COMPILER_LAUNCHER=ccache', '-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache', '-
DCMAKE_HIP_COMPILER_LAUNCHER=ccache', '-DVLLM_PYTHON_EXECUTABLE=/home/aiscuser/miniconda3/envs/myenveditablevllm085/bin/python3.12', '-DVLLM_PYTHON_PATH=/tmp/pip-build-env-wvein36i/site:/home/aiscuser/miniconda3/envs/myenveditablevllm085/lib/python312.zip:/home/aiscuser/miniconda3/envs/myenveditablevllm085/lib/python3.12:/home/aiscuser/minicond
a3/envs/myenveditablevllm085/lib/python3.12/lib-dynload:/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages:/tmp/pip-build-env-wvein36i/normal/lib/python3.12/site-packages:/tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/_vendor', '-DFETCHCONTENT_BASE_DIR=/scratch/AzureNfsServer_INPUT3/data/users/adatkins
/dev/phivnext/yoco/yocov2/vllm085_main/.deps', '-DNVCC_THREADS=128', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=64', '-DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc']' returned non-zero exit status 1.
  /tmp/pip-build-env-wvein36i/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py:1021: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `vllm` in editable mode.

Here's last few commits on main:

commit f4a8a3746575ea62fd3e0b4bd29ebd782582836d (HEAD -> main, origin/main, origin/HEAD)
Author: Michael Goin <mgoin64@gmail.com>
Date:   Tue May 20 12:08:37 2025 -0400

    [Minor] Rename quantization nvfp4 to modelopt_fp4 (#18356)

    Signed-off-by: mgoin <mgoin64@gmail.com>

commit 8f55962a7ffcf310ecb462f3a547e593e4fd77bd
Author: Reid <61492567+reidliu41@users.noreply.github.com>
Date:   Tue May 20 23:26:12 2025 +0800

    [Misc] refactor prompt embedding examples (#18405)

    Signed-off-by: reidliu41 <reid201711@gmail.com>
    Co-authored-by: reidliu41 <reid201711@gmail.com>

commit be48360c1fb9284804f9e1cae23b58e23e762877
Author: 燃 <wulipc@163.com>
Date:   Tue May 20 21:59:48 2025 +0800

    [Bugfix] Fix MRoPE Errors in the Qwen-VL Model When Processing Pure Text (#18407)

    Co-authored-by: 松灵 <wpf272043@alibaba-inc.com>

Encountered on A100 in Python=3.12 conda environment with cuda 12.4
cc @congcongchen123

[aatkinson]

Happens on cuda 12.6 too

(myenveditablevllm085post1_090pre) $ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Oct_29_23:50:19_PDT_2024
Cuda compilation tools, release 12.6, V12.6.85
Build cuda_12.6.r12.6/compiler.35059454_0
(myenveditablevllm085post1_090pre) $ nvidia-smi
Tue May 20 17:51:06 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 575.51.03              Driver Version: 575.51.03      CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A6000               Off |   00000000:01:00.0 Off |                  Off |
| 30%   39C    P8             24W /  300W |     103MiB /  49140MiB |      0%      Default |
|                                         |                        |                  N/A |

commit 980a172474fa0f32433dda87ae1fa4aadba24c51 (HEAD -> main, origin/main, origin/HEAD)
Author: Percy <xhc_1007@163.com>
Date:   Tue May 20 13:19:34 2025 -0500

    [Kernel] update comment for KV shape in unified triton attn (#18099)

    Signed-off-by: haochengxia <xhc_1007@163.com>

commit e1f5a71ed74b0e232e237070c14b50542ad15298
Author: Calvin Chen <45745657+calvin0327@users.noreply.github.com>
Date:   Wed May 21 00:40:05 2025 +0800

    [Model] use AutoWeightsLoader for bloom (#18300)

    Signed-off-by: calvin chen <120380290@qq.com>

commit f4a8a3746575ea62fd3e0b4bd29ebd782582836d
Author: Michael Goin <mgoin64@gmail.com>
Date:   Tue May 20 12:08:37 2025 -0400

    [Minor] Rename quantization nvfp4 to modelopt_fp4 (#18356)

    Signed-off-by: mgoin <mgoin64@gmail.com>

[congcongchen123]

Thanks @simon-mo , I also try out later cuda versions like cuda 12.4 or 12.6. And similar to @aatkinson , the build crashes as well. And the fix #17679 only mentions fixing on cuda 11.8. I thought that later cuda versions should not suffer from the same build failures, am I missing something?

[elvingerpaul]

I experienced the same error on H100, CUDA 12.5 and python 3.10.12

      -- Marlin generation script hash: 89eb88480f5ec8093709a8df88f872be
      -- Last run Marlin generate script hash:
      CMake Error at CMakeLists.txt:329 (message):
        Marlin generation failed.  Result: "1"
      
        Check the log for details:
        /tmp/tmpjqkrnaba.build-temp/marlin_generation.log

It was caused by running python3 csrc/quantization/gptq_marlin/generate_kernels.py.

/tmp/tmpjqkrnaba.build-temp/marlin_generation.log did also not exist for me. So I ran the script outside CMake to get the stacktrace. Turns out I was missing Jinja2.

So python3 -m pip install jinja2 and rebuilding vLLM from source solved it for me

[ccongcong321]

Ok, I am now able to build from source with CUDA 12.4 and python 3.12 and pytorch 2.5.

But I have to 1) run python3 -m pip install jinja2 and then 2) manually install the dependency: conda install mkl mkl-devel for below errors:

  [5/311] Linking CXX shared module cumem_allocator.abi3.so
  FAILED: cumem_allocator.abi3.so
  : && /usr/bin/c++ -fPIC -O2 -g -DNDEBUG   -shared  -o cumem_allocator.abi3.so CMakeFiles/cumem_allocator.dir/csrc/cumem_allocator.cpp.o -L/opt/conda/envs/ptca/lib   -L/opt/conda/envs/ptca/lib/intel64   -L/opt/conda/envs/ptca/lib/intel64_win   -L/opt/conda/envs/ptca/lib/win-x64 -Wl,-rpath,/opt/conda/envs/ptca/lib:/opt/conda/envs/ptca/lib/intel64:/opt/conda/envs/ptca/lib/intel64_win:/opt/conda/envs/ptca/lib/win-x64:/tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib:/usr/local/cuda/lib64:  /usr/lib/x86_64-linux-gnu/libcuda.so  /tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libtorch.so  /tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libc10.so  /usr/local/cuda/lib64/libnvrtc.so  /tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libc10_cuda.so  -Wl,--no-as-needed,"/tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so" -Wl,--as-needed  -lmkl_intel_ilp64  -lmkl_core  -lmkl_intel_thread  -Wl,--no-as-needed,"/tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libtorch_cuda.so" -Wl,--as-needed  /tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libc10_cuda.so  /tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libc10.so  /usr/local/cuda/lib64/libcudart.so  -Wl,--no-as-needed,"/tmp/pip-build-env-b1kw0_g6/overlay/lib/python3.12/site-packages/torch/lib/libtorch.so" -Wl,--as-needed  /usr/local/cuda/lib64/libnvToolsExt.so && :
  /usr/bin/ld: cannot find -lmkl_intel_ilp64: No such file or directory
  /usr/bin/ld: cannot find -lmkl_core: No such file or directory
  /usr/bin/ld: cannot find -lmkl_intel_thread: No such file or directory
  collect2: error: ld returned 1 exit status