build: bump torch to 2.7.1 and CUDA 12.8 support #1182

jim60105 · 2025-07-03T11:39:49Z

Pin Python version: Pin Python to 3.11 for uv default reference. It is not a restriction, but merely an added suggestion. Please refer to the official documentation.
Update PyTorch ecosystem: Bump PyTorch from 2.5.1 to 2.7.1 and switch to CUDA 12.8 support (cu128 index). The main reason is to avoid this 9.3 CRITICAL CVE-2025-32434, as this issue does indeed expose us to risk.
In addition, upgrading to cu128 means we theoretically support Blackwell (RTX 50XX), but I don't have the hardware to test it.
(Maybe) Add ARM64 support: Include triton ≥3.3.0 dependency for ARM64 architecture compatibility. Only tested in theory by packaging dependencies; I don't actually have an ARM machine with a graphics card to run it.
Constrain package versions: Lock numpy, onnxruntime, and pandas to specific ranges for Python 3.9 compatibility
Optimize package sources: Configure PyTorch packages to download from official PyTorch index while other packages use PyPI. This allowed me to avoid the issue previously mentioned where the new version of numpy drops support for Python 3.9. Although I found a compatible version, there is no guarantee regarding its stability.
Update documentation: Add CUDA installation instructions and remove obsolete cuDNN troubleshooting section , Resolves Can't get cuda to work. #983 since (I believe) this PR has handled the dependencies correctly.
Upgrade ctranslate2: Update minimum version from <4.5.0 to ≥4.5.0 , Resolves Upgrading ctranslate to >= 4.5.0 #1158, Resolves Issue with ctranslate2 package dependency #1190

This is a reissue of #1098 and I mentioned it in the closing comment of PR #1133.
This version of the code is currently released through the https://github.com/jim60105/docker-whisperX project.

- Add a .python-version file to pin the project to Python 3.11 - Update README setup to require CUDA toolkit 12.8 instead of 12.4 (Linux and Windows) - Raise the project’s minimum Python requirement to ≥3.10,<3.13 - Bump torch dependency from 2.6.0 to 2.7.1 - Switch the PyTorch CUDA wheel index from cu124 to cu128 Signed-off-by: CHEN, CHUN <[email protected]>

…n README" This reverts commit 6fe0a87. The issue of relying on two different versions of CUDNN in this project has been resolved. Signed-off-by: CHEN, CHUN <[email protected]>

- Only download torch from PyTorch; obtain all other packages from PyPI. There is a chance it can run on Python 3.9. - Restrict numpy, onnxruntime, pandas to be compatible with Python 3.9 Signed-off-by: CHEN, CHUN <[email protected]>

- Add triton version 3.3.0 or newer to the dependencies to support arm64 architecture. Signed-off-by: CHEN, CHUN <[email protected]>

fhalde · 2025-07-05T11:27:50Z

My understanding is that PyTorch comes pre-built with the required CUDA runtime libraries included. I’m a bit confused then, who are the official CUDA installation instructions meant for in that case? Thanks!

jim60105 · 2025-07-05T12:54:37Z

Doc from Pytorch

https://pytorch.org/get-started/locally/

Doc from NVIDIA

Then on a native Linux workstation with Podman, you probably work with CDI.

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/cdi-support.html

That's a lot. I always have Windows users install NV's CUDA Toolkit exe, and let Linux users read that document. (...Linux users probably know what they should do.)
And within the pip scope, get the source through PyTorch index to avoid compatibility issues.
In fact, pip cannot obtain all CUDA dependencies; I have encountered this issue in other projects.
In my experience, these instructions are the most reliable.

wywwzjj · 2025-07-06T04:27:42Z

Thanks

pyproject.toml

Saicheg · 2025-07-16T09:05:28Z

Really looking forward for this to be merged in order to upgrade my infrastructure!

- Add a platform marker to the triton dependency to skip it on Windows, as triton does not support Windows. Signed-off-by: Jim Chen <[email protected]>

Signed-off-by: Jim Chen <[email protected]>

kalvin807 · 2025-07-17T12:47:55Z

❯ uv add git+https://github.com/m-bain/whisperx --rev 251602f1220dc21d336d46eb1ebe100da74813cf
Using CPython 3.11.11
Creating virtual environment at: .venv
Resolved 123 packages in 754ms
error: Distribution `torch==2.7.1+cu128 @ registry+https://download.pytorch.org/whl/cu128` can't be installed because it doesn't have a source distribution or wheel for the current platform

hint: You're on macOS (`macosx_15_0_arm64`), but `torch` (v2.7.1+cu128) only has wheels for the following platforms: `manylinux_2_28_aarch64`, `manylinux_2_28_x86_64`, `win_amd64`; consider adding your platform to `tool.uv.required-environments` to ensure uv resolves to a version with compatible wheels

Thanks for the patch!

It does not install in macos via git install.
I believes it need extra setting in uv to tell uv install cpu pytorch in mac -> https://docs.astral.sh/uv/guides/integration/pytorch/#configuring-accelerators-with-optional-dependencies

- macOS uses CPU-only PyTorch from pytorch-cpu index - Linux and Windows use CUDA 12.8 PyTorch from pytorch index - triton only installs on Linux with CUDA 12.8 support - Update lockfile to support multi-platform builds

jim60105 · 2025-07-17T13:55:12Z

@kalvin807 Please try using 0e7153b to see if it works on Mac.

kalvin807 · 2025-07-17T15:09:18Z

it installed successfully. thanks

humblenginr · 2025-08-16T09:24:27Z

Thanks a bunch @jim60105 ! It works for me with cuda 12.8, torch 2.7.1 and cudnn 9.10.

I hope this gets merged soon @m-bain @seset

jim60105 added 4 commits July 3, 2025 05:48

Revert "docs: add troubleshooting section for libcudnn dependencies i…

640d642

…n README" This reverts commit 6fe0a87. The issue of relying on two different versions of CUDNN in this project has been resolved. Signed-off-by: CHEN, CHUN <[email protected]>

build(pyproject): require triton 3.3.0+ for arm64 support

327fa85

- Add triton version 3.3.0 or newer to the dependencies to support arm64 architecture. Signed-off-by: CHEN, CHUN <[email protected]>

jim60105 changed the title ~~build: update dependencies and Python version requirements~~ build: bump torch to 2.7.1 and CUDA 12.8 support Jul 3, 2025

jim60105 mentioned this pull request Jul 3, 2025

Upgrading ctranslate to >= 4.5.0 #1158

Open

seset reviewed Jul 16, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

jim60105 added 2 commits July 17, 2025 15:14

build: skip Triton on Windows since it isn't supported

835f4f4

- Add a platform marker to the triton dependency to skip it on Windows, as triton does not support Windows. Signed-off-by: Jim Chen <[email protected]>

build: uv lock --upgrade

251602f

Signed-off-by: Jim Chen <[email protected]>

build: configure PyTorch sources for cross-platform compatibility

0e7153b

- macOS uses CPU-only PyTorch from pytorch-cpu index - Linux and Windows use CUDA 12.8 PyTorch from pytorch index - triton only installs on Linux with CUDA 12.8 support - Update lockfile to support multi-platform builds

jim60105 mentioned this pull request Jul 20, 2025

Issue with ctranslate2 package dependency #1190

Open

jim60105 mentioned this pull request Jul 28, 2025

Could not load library libcudnn_ops_infer.so.8. on newest version. #967

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

build: bump torch to 2.7.1 and CUDA 12.8 support #1182

build: bump torch to 2.7.1 and CUDA 12.8 support #1182

Uh oh!

jim60105 commented Jul 3, 2025 •

edited

Loading

Uh oh!

fhalde commented Jul 5, 2025

Uh oh!

jim60105 commented Jul 5, 2025

Uh oh!

wywwzjj commented Jul 6, 2025

Uh oh!

Uh oh!

Saicheg commented Jul 16, 2025

Uh oh!

kalvin807 commented Jul 17, 2025

Uh oh!

jim60105 commented Jul 17, 2025

Uh oh!

kalvin807 commented Jul 17, 2025

Uh oh!

humblenginr commented Aug 16, 2025

Uh oh!

Uh oh!

Uh oh!

build: bump torch to 2.7.1 and CUDA 12.8 support #1182

Are you sure you want to change the base?

build: bump torch to 2.7.1 and CUDA 12.8 support #1182

Uh oh!

Conversation

jim60105 commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fhalde commented Jul 5, 2025

Uh oh!

jim60105 commented Jul 5, 2025

Doc from Pytorch

Doc from NVIDIA

Windows

Windows WSL

Linux, OSX

Uh oh!

wywwzjj commented Jul 6, 2025

Uh oh!

Uh oh!

Saicheg commented Jul 16, 2025

Uh oh!

kalvin807 commented Jul 17, 2025

Uh oh!

jim60105 commented Jul 17, 2025

Uh oh!

kalvin807 commented Jul 17, 2025

Uh oh!

humblenginr commented Aug 16, 2025

Uh oh!

Uh oh!

jim60105 commented Jul 3, 2025 •

edited

Loading