[Bug] TurboMind能否在windows下运行？

[jiangxinufo]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

目前来看是不能的，有没有什么方案在windows上运行TurboMind推理引擎？

Reproduction

目前来看是不能的，有没有什么方案在windows上运行TurboMind推理引擎？

Environment

目前来看是不能的，有没有什么方案在windows上运行TurboMind推理引擎？

Error traceback

[windreamer]

TurboMind backend can run on Windows System. So can I know what issue are you facing?

[jiangxinufo]

操作系统：

windows server 2022 21h2

环境依赖：

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> conda list
# packages in environment at F:\CondaData\envs\lmdeploy:
#
# Name                    Version                   Build  Channel
accelerate                1.10.1                   pypi_0    pypi
addict                    2.4.0                    pypi_0    pypi
aiohappyeyeballs          2.6.1                    pypi_0    pypi
aiohttp                   3.13.0                   pypi_0    pypi
aiosignal                 1.4.0                    pypi_0    pypi
annotated-types           0.7.0                    pypi_0    pypi
anyio                     4.11.0                   pypi_0    pypi
async-timeout             5.0.1                    pypi_0    pypi
attrs                     25.4.0                   pypi_0    pypi
bzip2                     1.0.8                h2466b09_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ca-certificates           2022.10.11           haa95532_0    https://mirrors.aliyun.com/anaconda/pkgs/main
certifi                   2025.10.5                pypi_0    pypi
charset-normalizer        3.4.3                    pypi_0    pypi
click                     8.3.0                    pypi_0    pypi
cloudpickle               3.1.1                    pypi_0    pypi
cmake                     4.1.0                    pypi_0    pypi
cmake-build-extension     0.6.1                    pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
datasets                  4.1.1                    pypi_0    pypi
dill                      0.4.0                    pypi_0    pypi
diskcache                 5.6.3                    pypi_0    pypi
distro                    1.9.0                    pypi_0    pypi
einops                    0.8.1                    pypi_0    pypi
exceptiongroup            1.3.0                    pypi_0    pypi
fastapi                   0.118.2                  pypi_0    pypi
filelock                  3.20.0                   pypi_0    pypi
fire                      0.7.1                    pypi_0    pypi
frozenlist                1.8.0                    pypi_0    pypi
fsspec                    2025.9.0                 pypi_0    pypi
genson                    1.3.0                    pypi_0    pypi
gitdb                     4.0.12                   pypi_0    pypi
gitpython                 3.1.45                   pypi_0    pypi
h11                       0.16.0                   pypi_0    pypi
httpcore                  1.0.9                    pypi_0    pypi
httpx                     0.28.1                   pypi_0    pypi
huggingface-hub           0.35.3                   pypi_0    pypi
idna                      3.10                     pypi_0    pypi
intel-openmp              2021.4.0                 pypi_0    pypi
jinja2                    3.1.6                    pypi_0    pypi
jiter                     0.11.0                   pypi_0    pypi
jsonpath-ng               1.7.0                    pypi_0    pypi
jsonschema                4.25.1                   pypi_0    pypi
jsonschema-specifications 2025.9.1                 pypi_0    pypi
libffi                    3.4.6                h537db12_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
liblzma                   5.6.4                h2466b09_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libsqlite                 3.49.1               h67fdade_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libzlib                   1.3.1                h2466b09_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
lmdeploy                  0.10.1                   pypi_0    pypi
markdown-it-py            4.0.0                    pypi_0    pypi
markupsafe                3.0.3                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mkl                       2021.4.0                 pypi_0    pypi
mmengine-lite             0.10.7                   pypi_0    pypi
modelscope                1.30.0                   pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
msgpack                   1.1.2                    pypi_0    pypi
multidict                 6.7.0                    pypi_0    pypi
multiprocess              0.70.16                  pypi_0    pypi
networkx                  3.4.2                    pypi_0    pypi
ninja                     1.13.0                   pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
nvidia-ml-py              13.580.82                pypi_0    pypi
nvidia-ml-py3             7.352.0                  pypi_0    pypi
openai                    2.2.0                    pypi_0    pypi
openai-harmony            0.0.4                    pypi_0    pypi
openssl                   3.4.1                ha4e3fda_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
outlines                  1.2.5                    pypi_0    pypi
outlines-core             0.2.11                   pypi_0    pypi
packaging                 25.0                     pypi_0    pypi
pandas                    2.3.3                    pypi_0    pypi
partial-json-parser       0.2.1.1.post6            pypi_0    pypi
peft                      0.14.0                   pypi_0    pypi
pillow                    11.3.0                   pypi_0    pypi
pip                       25.0.1             pyh8b19718_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
platformdirs              4.5.0                    pypi_0    pypi
ply                       3.11                     pypi_0    pypi
prometheus-client         0.23.1                   pypi_0    pypi
propcache                 0.4.1                    pypi_0    pypi
protobuf                  6.32.1                   pypi_0    pypi
psutil                    7.1.0                    pypi_0    pypi
pyarrow                   21.0.0                   pypi_0    pypi
pybind11                  3.0.1                    pypi_0    pypi
pydantic                  2.12.0                   pypi_0    pypi
pydantic-core             2.41.1                   pypi_0    pypi
pygments                  2.19.2                   pypi_0    pypi
pynvml                    13.0.1                   pypi_0    pypi
python                    3.10.16         h37870fc_1_cpython    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
pytz                      2025.2                   pypi_0    pypi
pyyaml                    6.0.3                    pypi_0    pypi
pyzmq                     27.1.0                   pypi_0    pypi
ray                       2.49.2                   pypi_0    pypi
referencing               0.36.2                   pypi_0    pypi
regex                     2025.9.18                pypi_0    pypi
requests                  2.32.5                   pypi_0    pypi
rich                      14.1.0                   pypi_0    pypi
rpds-py                   0.27.1                   pypi_0    pypi
safetensors               0.6.2                    pypi_0    pypi
sentencepiece             0.2.1                    pypi_0    pypi
setuptools                80.9.0                   pypi_0    pypi
setuptools-scm            9.2.0                    pypi_0    pypi
shortuuid                 1.0.13                   pypi_0    pypi
six                       1.17.0                   pypi_0    pypi
smmap                     5.0.2                    pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
starlette                 0.48.0                   pypi_0    pypi
sympy                     1.14.0                   pypi_0    pypi
tbb                       2021.13.1                pypi_0    pypi
termcolor                 3.1.0                    pypi_0    pypi
tiktoken                  0.12.0                   pypi_0    pypi
tk                        8.6.13               h5226925_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tokenizers                0.22.1                   pypi_0    pypi
tomli                     2.3.0                    pypi_0    pypi
torch                     2.3.1+cu118              pypi_0    pypi
torchaudio                2.3.1+cu118              pypi_0    pypi
torchvision               0.18.1+cu118             pypi_0    pypi
tqdm                      4.67.1                   pypi_0    pypi
transformers              4.57.0                   pypi_0    pypi
transformers-stream-generator 0.0.5                    pypi_0    pypi
triton                    3.0.0                    pypi_0    pypi
triton-windows            3.4.0.post20             pypi_0    pypi
typing-extensions         4.15.0                   pypi_0    pypi
typing-inspection         0.4.2                    pypi_0    pypi
tzdata                    2025.2                   pypi_0    pypi
ucrt                      10.0.22621.0         h57928b3_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
urllib3                   2.5.0                    pypi_0    pypi
uvicorn                   0.37.0                   pypi_0    pypi
vc                        14.3                h5fd82a7_24    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
vc14_runtime              14.42.34433         h6356254_24    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
wheel                     0.45.1                   pypi_0    pypi
xxhash                    3.6.0                    pypi_0    pypi
yapf                      0.43.0                   pypi_0    pypi
yarl                      1.22.0                   pypi_0    pypi

报错内容：

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> python
Python 3.10.16 | packaged by conda-forge | (main, Dec 5 2024, 14:07:43) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>from lmdeploy import pipeline
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda_*init*_.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml # type: ignore[import]
>>>pipe =pipeline('C:\Users\Administrator\.cache\modelscope\hub\models\Shanghai_AI_Laboratory\internlm2_5-7b-chat')
2025-10-10 10:26:47,628 - lmdeploy - WARNING - archs.py:49 - Fallback to pytorch engine because turbomind engine is not installed correctly. If you insist to use turbomind engine, you may need to reinstall lmdeploy from pypi or build from source and try again.
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda_*init*_.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml # type: ignore[import]

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "", line 1, in
File "F:\CondaData\envs\lmdeploy\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "F:\CondaData\envs\lmdeploy\lib\multiprocessing\spawn.py", line 129, in _main
return self._bootstrap(parent_sentinel)
File "F:\CondaData\envs\lmdeploy\lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "F:\CondaData\envs\lmdeploy\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "F:\CondaData\envs\lmdeploy\lib\concurrent\futures\process.py", line 246, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\engine\engine_checker.py", line 101, in _handle_impl
return super().handle()
File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\base.py", line 41, in handle
checker.handle()
File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\base.py", line 41, in handle
checker.handle()
File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\base.py", line 41, in handle
checker.handle()
File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\base.py", line 42, in handle
self.check()
File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\torch.py", line 16, in check
a = torch.tensor([1, 2], device=self.device)
F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\torch.py:16: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
a = torch.tensor([1, 2], device=self.device)
2025-10-10 10:27:21,520 - lmdeploy - ERROR - base.py:53 - RuntimeError: Failed to find C compiler. Please specify via CC environment variable.
2025-10-10 10:27:21,521 - lmdeploy - ERROR - base.py:54 - check failed!
Please ensure that your device is functioning properly with .
You can verify your environment by running `python -m lmdeploy.pytorch.check_env.triton_custom_add`.

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> python -m lmdeploy.pytorch.check_env.triton_custom_add
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda\__init__.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 29, in <module>
    a = torch.tensor([1, 2], device='cuda')
F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py:29: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
  a = torch.tensor([1, 2], device='cuda')
Traceback (most recent call last):
  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 31, in <module>
    c = custom_add(a, b)
  File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 24, in custom_add
    _add_kernel[grid](a, b, c, size, BLOCK=BLOCK)
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\jit.py", line 166, in <lambda>
    return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\jit.py", line 348, in run
    device = driver.get_current_device()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 230, in __getattr__
    self._initialize_obj()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 227, in _initialize_obj
    self._obj = self._init_fn()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 260, in initialize_driver
    return CudaDriver()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 122, in __init__
    self.utils = CudaUtils()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 69, in __init__
    so = _build("cuda_utils", src_path, tmpdir)
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\common\build.py", line 101, in _build
    raise RuntimeError("Failed to find C compiler. Please specify via CC environment variable.")
RuntimeError: Failed to find C compiler. Please specify via CC environment variable.

当我安装完pip install -U "triton-windows<3.5"之后不再报错，但是提示“2025-10-10 13:25:04,630 - lmdeploy - WARNING - archs.py:49 - Fallback to pytorch engine because turbomind engine is not installed correctly. If you insist to use turbomind engine, you may need to reinstall lmdeploy from pypi or build from source and try again.”

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> python
Python 3.10.16 | packaged by conda-forge | (main, Dec  5 2024, 14:07:43) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

from lmdeploy import pipeline
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda\__init__.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml  # type: ignore[import]
pipe = pipeline('C:\\Users\\Administrator\\.cache\\modelscope\\hub\\models\\Shanghai_AI_Laboratory\\internlm2_5-7b-chat')
2025-10-10 13:25:04,630 - lmdeploy - WARNING - archs.py:49 - Fallback to pytorch engine because turbomind engine is not installed correctly. If you insist to use turbomind engine, you may need to reinstall lmdeploy from pypi or build from source and try again.
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda\__init__.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml  # type: ignore[import]
2025-10-10 13:25:39,443 - lmdeploy - WARNING - transformers.py:22 - LMDeploy requires transformers version: [4.33.0 ~ 4.56.1], but found version: 4.57.0
2025-10-10 13:25:44,699 - lmdeploy - WARNING - attention.py:229 - For higher performance, please install FlashAttention-3 https://github.com/Dao-AILab/flash-attention
Loading weights from safetensors: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:11<00:00,  1.45s/it]
response = pipe(['Hi, pls intro yourself', 'Shanghai is'])
2025-10-10 13:26:58,279 - lmdeploy - WARNING - tokenizer.py:471 - Detected duplicate bos token 1 in prompt, this will likely reduce response quality, one of them will beremoved
print(response)
[text=Hello! I'm a helpful assistant designed to provide information, answer questions, and assist with various tasks. I don't have personal experiences or emotions, but I'm here to make
 your interactions as smooth and informative as possible. Whether you need help with a specific topic, have questions about technology, or need advice on a wide range of subjects, feel free to ask!
generate_token_len=74
finish_reason="stop"
token_ids=[9843, 346, 489, 2940, 395, 11100, 17993, 6342, 442, 3572, 2145, 328, 4384, 4917, 328, 454, 7928, 579, 5417, 9208, 281, 489, 1658, 1082, 746, 4505, 11599, 607, 21485, 328, 847,
 489, 2940, 1734, 442, 1426, 829, 22140, 569, 11026, 454, 38061, 569, 3369, 281, 13281, 629, 1329, 1638, 579, 395, 3317, 8677, 328, 746, 4917, 1048, 5599, 328, 607, 1329, 9591, 519, 395, 7037, 2240, 446, 15007, 328, 2828, 2060, 442, 2705, 346]
log_probs=None
logits=None
last_hidden_state=None, text=Shanghai is a vibrant and dynamic city located in the eastern part of China. It is the largest city in terms of both area and population in China, and it serves as the economic, financial, and cultural center of the country. Shanghai is known for its rich history, stunning skyline, and its role as a global city.

Key features and attractions of Shanghai include:

1. **The Bund**: A historic waterfront area that offers a stunning view of the city's skyline, featuring a mix of colonial-era buildings and modern skyscrapers.

2. **Yu Garden**: A classical Chinese garden that showcases the art of traditional Chinese landscape design and architecture.

3. **Shanghai Tower**: One of the tallest buildings in the world, offering breathtaking views of the city from its observation deck.

4. **The Oriental Pearl Tower**: A prominent landmark of Shanghai, featuring a unique design resembling a cluster of pearls.

5. **Nanjing Road**: A bustling shopping street that is one of the most famous in Asia, offering a wide range of retail options from luxury brands to local boutiques.

6. **Shanghai Museum**: A world-class museum that houses an extensive collection of Chinese art and artifacts, including ancient ceramics, jade, and calligraphy.

7. **Pudong**: A modern district on the eastern side of the Huangpu River, known for its futuristic architecture, including the Shanghai World Financial Center and the Shanghai Grand Theatre.

Shanghai is also a major transportation hub, with two international airports (Pudong International Airport and Shanghai虹桥国际机场), an extensive subway system, and a network of highways and bridges connecting the city to other parts of China and beyond.

The city's economic importance is underscored by its position as a global financial center, home to the Shanghai Stock Exchange and the China Financial Center. Additionally, Shanghai is a key player in international trade, hosting numerous conventions, exhibitions, and events that attract visitors from around the world.

In summary, Shanghai is a city that seamlessly blends its storied past with a forward-looking future, offering a unique blend of cultural heritage, modern amenities, and economic dynamism.
generate_token_len=453
finish_reason="stop"
token_ids=[2166, 30326, 505, 395, 33083, 454, 8878, 3446, 7553, 435, 410, 23478, 1087, 446, 5772, 281, 1226, 505, 410, 7911, 3446, 435, 3949, 446, 2329, 3247, 454, 7192, 435, 5772, 328, 
454, 563, 17186, 569, 410, 7105, 328, 6050, 328, 454, 12901, 4285, 446, 410, 3311, 281, 36956, 505, 4037, 500, 1326, 9209, 3995, 328, 20026, 13034, 1198, 328, 454, 1326, 3638, 569, 395, 
3805, 3446, 512, 1738, 4581, 454, 38322, 446, 36956, 3089, 1593, 312, 281, 3235, 918, 29887, 465, 334, 493, 18183, 3181, 7101, 3247, 560, 6234, 395, 20026, 1800, 446, 410, 3446, 725, 130
34, 1198, 328, 16580, 395, 6671, 446, 36518, 50501, 13841, 454, 6637, 2051, 1189, 271, 4772, 518, 512, 314, 281, 3235, 337, 275, 19175, 465, 334, 493, 29145, 8588, 13693, 560, 1620, 2428
0, 410, 2099, 446, 8740, 8588, 18566, 3047, 454, 17786, 512, 308, 281, 3235, 2166, 30326, 22202, 465, 334, 3936, 446, 410, 390, 19399, 13841, 435, 410, 2028, 328, 10140, 55015, 6348, 446
, 410, 3446, 635, 1326, 22194, 9662, 512, 319, 281, 3235, 918, 37764, 6453, 36187, 22202, 465, 334, 493, 20657, 37146, 446, 36956, 328, 16580, 395, 5069, 3047, 443, 41643, 395, 10796, 44
6, 37561, 4892, 512, 317, 281, 3235, 304, 51217, 418, 9668, 465, 334, 493, 20988, 2880, 12069, 8725, 560, 505, 959, 446, 410, 1575, 11396, 435, 13764, 328, 10140, 395, 7037, 2240, 446, 1

[jiangxinufo]

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> python -m torch.utils.collect_env                                                                                 
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda\__init__.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
F:\CondaData\envs\lmdeploy\lib\runpy.py:126: RuntimeWarning: 'torch.utils.collect_env' found in sys.modules after import of package 'torch.utils', but prior to execution of 'torch.utils.collect_env'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Collecting environment information...
PyTorch version: 2.3.1+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Microsoft Windows Server 2022 Standard
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 4.1.0
Libc version: N/A

Python version: 3.10.16 | packaged by conda-forge | (main, Dec  5 2024, 14:07:43) [MSC v.1942 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.20348-SP0
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=2095
DeviceID=CPU0
Family=179
L2CacheSize=20480
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2095
Name=Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz
ProcessorType=3
Revision=21767


Architecture=9
CurrentClockSpeed=2095
DeviceID=CPU1
Family=179
L2CacheSize=20480
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2095
Name=Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz
ProcessorType=3
Revision=21767

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.3.1+cu118
[pip3] torchaudio==2.3.1+cu118
[pip3] torchvision==0.18.1+cu118
[pip3] triton==3.0.0
[pip3] triton-windows==3.4.0.post20
[conda] mkl                       2021.4.0                 pypi_0    pypi
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] torch                     2.3.1+cu118              pypi_0    pypi
[conda] torchaudio                2.3.1+cu118              pypi_0    pypi
[conda] torchvision               0.18.1+cu118             pypi_0    pypi
[conda] triton                    3.0.0                    pypi_0    pypi
[conda] triton-windows            3.4.0.post20             pypi_0    pypi

[windreamer]

I noticed this line in the log:

2025-10-10 13:25:04,630 - lmdeploy - WARNING - archs.py:49 - Fallback to pytorch engine because turbomind engine is not installed correctly. If you insist to use turbomind engine, you may need to reinstall lmdeploy from pypi or build from source and try again.

It suggests that we can not import turbomind's so/dll successfully. So can you kind examine if we have the so/dll file in the path? I think the right path should be F:\CondaData\envs\lmdeploy\lib\site-packages\lmdeploy\lib

[jiangxinufo]

只有一个pyd文件，我该如何处理呢

[windreamer]

只有一个pyd文件，我该如何处理呢

那么这应该是正常的

我也注意到你的环境里可能有两个 lmdeploy：
第一个是 pip 安装的 wheel 包，位于
F:\CondaData\envs\lmdeploy\lib\site-packages\lmdeploy
第二个看起来是源码目录，位于
F:\jx\lmdeploy\lmdeploy-main

如果是这样，我建议你把后者删掉，否则可能会出现冲突。因为源码目录没有编译的话是没有turbomind的

[jiangxinufo]

抱歉，我只看到了一个lmdeploy包，另一个在哪，谢谢！

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> conda list
# packages in environment at F:\CondaData\envs\lmdeploy:
#
# Name                    Version                   Build  Channel
accelerate                1.10.1                   pypi_0    pypi
addict                    2.4.0                    pypi_0    pypi
aiohappyeyeballs          2.6.1                    pypi_0    pypi
aiohttp                   3.13.0                   pypi_0    pypi
aiosignal                 1.4.0                    pypi_0    pypi
annotated-types           0.7.0                    pypi_0    pypi
anyio                     4.11.0                   pypi_0    pypi
async-timeout             5.0.1                    pypi_0    pypi
attrs                     25.4.0                   pypi_0    pypi
bzip2                     1.0.8                h2466b09_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ca-certificates           2022.10.11           haa95532_0    https://mirrors.aliyun.com/anaconda/pkgs/main
certifi                   2025.10.5                pypi_0    pypi
charset-normalizer        3.4.3                    pypi_0    pypi
click                     8.3.0                    pypi_0    pypi
cloudpickle               3.1.1                    pypi_0    pypi
cmake                     4.1.0                    pypi_0    pypi
cmake-build-extension     0.6.1                    pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
datasets                  4.1.1                    pypi_0    pypi
dill                      0.4.0                    pypi_0    pypi
diskcache                 5.6.3                    pypi_0    pypi
distro                    1.9.0                    pypi_0    pypi
einops                    0.8.1                    pypi_0    pypi
exceptiongroup            1.3.0                    pypi_0    pypi
fastapi                   0.118.2                  pypi_0    pypi
filelock                  3.20.0                   pypi_0    pypi
fire                      0.7.1                    pypi_0    pypi
frozenlist                1.8.0                    pypi_0    pypi
fsspec                    2025.9.0                 pypi_0    pypi
genson                    1.3.0                    pypi_0    pypi
gitdb                     4.0.12                   pypi_0    pypi
gitpython                 3.1.45                   pypi_0    pypi
h11                       0.16.0                   pypi_0    pypi
httpcore                  1.0.9                    pypi_0    pypi
httpx                     0.28.1                   pypi_0    pypi
huggingface-hub           0.35.3                   pypi_0    pypi
idna                      3.10                     pypi_0    pypi
intel-openmp              2021.4.0                 pypi_0    pypi
jinja2                    3.1.6                    pypi_0    pypi
jiter                     0.11.0                   pypi_0    pypi
jsonpath-ng               1.7.0                    pypi_0    pypi
jsonschema                4.25.1                   pypi_0    pypi
jsonschema-specifications 2025.9.1                 pypi_0    pypi
libffi                    3.4.6                h537db12_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
liblzma                   5.6.4                h2466b09_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libsqlite                 3.49.1               h67fdade_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libzlib                   1.3.1                h2466b09_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
lmdeploy                  0.10.1                   pypi_0    pypi
markdown-it-py            4.0.0                    pypi_0    pypi
markupsafe                3.0.3                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mkl                       2021.4.0                 pypi_0    pypi
mmengine-lite             0.10.7                   pypi_0    pypi
modelscope                1.30.0                   pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
msgpack                   1.1.2                    pypi_0    pypi
multidict                 6.7.0                    pypi_0    pypi
multiprocess              0.70.16                  pypi_0    pypi
networkx                  3.4.2                    pypi_0    pypi
ninja                     1.13.0                   pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
nvidia-ml-py              13.580.82                pypi_0    pypi
nvidia-ml-py3             7.352.0                  pypi_0    pypi
openai                    2.2.0                    pypi_0    pypi
openai-harmony            0.0.4                    pypi_0    pypi
openssl                   3.4.1                ha4e3fda_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
outlines                  1.2.5                    pypi_0    pypi
outlines-core             0.2.11                   pypi_0    pypi
packaging                 25.0                     pypi_0    pypi
pandas                    2.3.3                    pypi_0    pypi
partial-json-parser       0.2.1.1.post6            pypi_0    pypi
peft                      0.14.0                   pypi_0    pypi
pillow                    11.3.0                   pypi_0    pypi
pip                       25.0.1             pyh8b19718_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
platformdirs              4.5.0                    pypi_0    pypi
ply                       3.11                     pypi_0    pypi
prometheus-client         0.23.1                   pypi_0    pypi
propcache                 0.4.1                    pypi_0    pypi
protobuf                  6.32.1                   pypi_0    pypi
psutil                    7.1.0                    pypi_0    pypi
pyarrow                   21.0.0                   pypi_0    pypi
pybind11                  3.0.1                    pypi_0    pypi
pydantic                  2.12.0                   pypi_0    pypi
pydantic-core             2.41.1                   pypi_0    pypi
pygments                  2.19.2                   pypi_0    pypi
pynvml                    13.0.1                   pypi_0    pypi
python                    3.10.16         h37870fc_1_cpython    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
pytz                      2025.2                   pypi_0    pypi
pyyaml                    6.0.3                    pypi_0    pypi
pyzmq                     27.1.0                   pypi_0    pypi
ray                       2.49.2                   pypi_0    pypi
referencing               0.36.2                   pypi_0    pypi
regex                     2025.9.18                pypi_0    pypi
requests                  2.32.5                   pypi_0    pypi
rich                      14.1.0                   pypi_0    pypi
rpds-py                   0.27.1                   pypi_0    pypi
safetensors               0.6.2                    pypi_0    pypi
sentencepiece             0.2.1                    pypi_0    pypi
setuptools                80.9.0                   pypi_0    pypi
setuptools-scm            9.2.0                    pypi_0    pypi
shortuuid                 1.0.13                   pypi_0    pypi
six                       1.17.0                   pypi_0    pypi
smmap                     5.0.2                    pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
starlette                 0.48.0                   pypi_0    pypi
sympy                     1.14.0                   pypi_0    pypi
tbb                       2021.13.1                pypi_0    pypi
termcolor                 3.1.0                    pypi_0    pypi
tiktoken                  0.12.0                   pypi_0    pypi
tk                        8.6.13               h5226925_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tokenizers                0.22.1                   pypi_0    pypi
tomli                     2.3.0                    pypi_0    pypi
torch                     2.3.1+cu118              pypi_0    pypi
torchaudio                2.3.1+cu118              pypi_0    pypi
torchvision               0.18.1+cu118             pypi_0    pypi
tqdm                      4.67.1                   pypi_0    pypi
transformers              4.57.0                   pypi_0    pypi
transformers-stream-generator 0.0.5                    pypi_0    pypi
triton                    3.0.0                    pypi_0    pypi
triton-windows            3.4.0.post20             pypi_0    pypi
typing-extensions         4.15.0                   pypi_0    pypi
typing-inspection         0.4.2                    pypi_0    pypi
tzdata                    2025.2                   pypi_0    pypi
ucrt                      10.0.22621.0         h57928b3_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
urllib3                   2.5.0                    pypi_0    pypi
uvicorn                   0.37.0                   pypi_0    pypi
vc                        14.3                h5fd82a7_24    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
vc14_runtime              14.42.34433         h6356254_24    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
wheel                     0.45.1                   pypi_0    pypi
xxhash                    3.6.0                    pypi_0    pypi
yapf                      0.43.0                   pypi_0    pypi
yarl                      1.22.0                   pypi_0    pypi

[jiangxinufo]

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> pip show lmdeploy
Name: lmdeploy
Version: 0.10.1
Summary: A toolset for compressing, deploying and serving LLM
Home-page:
Author: OpenMMLab
Author-email: openmmlab@gmail.com
License:
Location: f:\condadata\envs\lmdeploy\lib\site-packages
Requires: accelerate, aiohttp, einops, fastapi, fire, mmengine-lite, numpy, openai, openai_harmony, outlines, partial_json_parser, peft, pillow, prometheus_client, protobuf, pydantic, pynvml, pyzmq, ray, safetensors, sentencepiece, shortuuid, tiktoken, torch, torchvision, transformers, uvicorn
Required-by:

[windreamer]

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> pip show lmdeploy
Name: lmdeploy
Version: 0.10.1
Summary: A toolset for compressing, deploying and serving LLM
Home-page:
Author: OpenMMLab
Author-email: openmmlab@gmail.com
License:
Location: f:\condadata\envs\lmdeploy\lib\site-packages
Requires: accelerate, aiohttp, einops, fastapi, fire, mmengine-lite, numpy, openai, openai_harmony, outlines, partial_json_parser, peft, pillow, prometheus_client, protobuf, pydantic, pynvml, pyzmq, ray, safetensors, sentencepiece, shortuuid, tiktoken, torch, torchvision, transformers, uvicorn
Required-by:

看起来这个 package 似乎是好的。
我觉得你在 F:\jx\lmdeploy\lmdeploy-main 这个目录下有一份 lmdeploy 的代码。从以下日志来看你是在用这份没有编译过的lmdeploy

(lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> python -m lmdeploy.pytorch.check_env.triton_custom_add
F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda\__init__.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 29, in <module>
    a = torch.tensor([1, 2], device='cuda')
F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py:29: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
  a = torch.tensor([1, 2], device='cuda')
Traceback (most recent call last):
  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 31, in <module>
    c = custom_add(a, b)
  File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 24, in custom_add
    _add_kernel[grid](a, b, c, size, BLOCK=BLOCK)
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\jit.py", line 166, in <lambda>
    return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\jit.py", line 348, in run
    device = driver.get_current_device()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 230, in __getattr__
    self._initialize_obj()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 227, in _initialize_obj
    self._obj = self._init_fn()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 260, in initialize_driver
    return CudaDriver()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 122, in __init__
    self.utils = CudaUtils()
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 69, in __init__
    so = _build("cuda_utils", src_path, tmpdir)
  File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\common\build.py", line 101, in _build
    raise RuntimeError("Failed to find C compiler. Please specify via CC environment variable.")
RuntimeError: Failed to find C compiler. Please specify via CC environment variable.

[jiangxinufo]

确实是，非常感谢！

…

---- 回复的原邮件 ---- | 发件人 | ***@***.***> | | 发送日期 | 2025年10月10日 16:01 | | 收件人 | ***@***.***> | | 抄送人 | ***@***.***>, ***@***.***> | | 主题 | Re: [InternLM/lmdeploy] [Bug] TurboMind能否在windows下运行？ (Issue #4031) | windreamer left a comment (InternLM/lmdeploy#4031) (lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> pip show lmdeploy Name: lmdeploy Version: 0.10.1 Summary: A toolset for compressing, deploying and serving LLM Home-page: Author: OpenMMLab Author-email: ***@***.*** License: Location: f:\condadata\envs\lmdeploy\lib\site-packages Requires: accelerate, aiohttp, einops, fastapi, fire, mmengine-lite, numpy, openai, openai_harmony, outlines, partial_json_parser, peft, pillow, prometheus_client, protobuf, pydantic, pynvml, pyzmq, ray, safetensors, sentencepiece, shortuuid, tiktoken, torch, torchvision, transformers, uvicorn Required-by: 看起来这个 package 似乎是好的。 我觉得你在 F:\jx\lmdeploy\lmdeploy-main 这个目录下有一份 lmdeploy 的代码。从以下日志来看你是在用这份没有编译过的lmdeploy (lmdeploy) PS F:\jx\lmdeploy\lmdeploy-main> python -m lmdeploy.pytorch.check_env.triton_custom_add F:\CondaData\envs\lmdeploy\lib\site-packages\torch\cuda\__init__.py:56: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. import pynvml # type: ignore[import] A module that was compiled using NumPy 1.x cannot be run in NumPy 2.2.6 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2. Traceback (most recent call last): File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 29, in <module> a = torch.tensor([1, 2], device='cuda') F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py:29: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.) a = torch.tensor([1, 2], device='cuda') Traceback (most recent call last): File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "F:\CondaData\envs\lmdeploy\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 31, in <module> c = custom_add(a, b) File "F:\jx\lmdeploy\lmdeploy-main\lmdeploy\pytorch\check_env\triton_custom_add.py", line 24, in custom_add _add_kernel[grid](a, b, c, size, BLOCK=BLOCK) File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\jit.py", line 166, in <lambda> return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs) File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\jit.py", line 348, in run device = driver.get_current_device() File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 230, in __getattr__ self._initialize_obj() File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 227, in _initialize_obj self._obj = self._init_fn() File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 260, in initialize_driver return CudaDriver() File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 122, in __init__ self.utils = CudaUtils() File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\runtime\driver.py", line 69, in __init__ so = _build("cuda_utils", src_path, tmpdir) File "F:\CondaData\envs\lmdeploy\lib\site-packages\triton\common\build.py", line 101, in _build raise RuntimeError("Failed to find C compiler. Please specify via CC environment variable.") RuntimeError: Failed to find C compiler. Please specify via CC environment variable. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>