FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

[minmie]

Describe the issue

when i inference with optimum.onnxruntime，an error occurred:

FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

my env is as follow:
cuda 12.1
cudnn8
onnxruntime-gpu 1.16

and I found this doc(https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html):

so I tried to update onnxruntime-gpu to 1.18.0, but the same error still occurred.

But when I updated the version of onnxruntime-gpu to 1.18.1, another error occurred， as follows：
FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.9: cannot open shared object file: No such file or directory

so， with cuda12.1 and cudnn8, which version of onnxruntime-gpu should i install？

To reproduce

pass

Urgency

not

Platform

Linux

OS Version

ubuntu 20.04.6 LTS

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

onnxruntime-gpu 1.18.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

cuda 12.1

[tianleiwu]

https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#cuda-12x

ONNX Runtime	CUDA	cuDNN
1.18.1	12.x	9.x
1.18.0	12.x	8.x

From the above table, 1.18.0 is good for cuda 12.x and cudnn 8.x.

You can install like

pip install onnxruntime-gpu==1.18.0 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/ --force-reinstall

See https://onnxruntime.ai/docs/install/#install-onnx-runtime-gpu-cuda-12x for installation.

[andreclaudino]

I am using

onnx                     1.16.2
onnxruntime-gpu          1.18.1

With cuda 12.x and cudnn 9.x, based on the provided version table, that would work. But I am stuck on this error requiring libcublasLt.so.11 wich does not exist in cuda 12.x instalation.

[eaven123]

I am using

onnx                     1.16.2
onnxruntime-gpu          1.18.1

With cuda 12.x and cudnn 9.x, based on the provided version table, that would work. But I am stuck on this error requiring wich does not exist in cuda 12.x instalation.libcublasLt.so.11

How did you finally solve it? I'm having the same problem too.

[tianleiwu]

I am using

onnx                     1.16.2
onnxruntime-gpu          1.18.1

With cuda 12.x and cudnn 9.x, based on the provided version table, that would work. But I am stuck on this error requiring libcublasLt.so.11 wich does not exist in cuda 12.x instalation.

@andreclaudino, @eaven123 , you can pip install onnxruntime-gpu==1.19.0 to work with cuda 12.x and cudnn 9.x.

If you pip install onnxruntime-gpu==1.18.1, you need cuda 11.8 and cudnn 8.9.

[eaven123]

okk，thanks

…

---- Replied Message ---- | From | Tianlei ***@***.***> | | Date | 08/21/2024 11:44 | | To | microsoft/onnxruntime ***@***.***> | | Cc | eaven123 ***@***.***>, Comment ***@***.***> | | Subject | Re: [microsoft/onnxruntime] FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory (Issue #21684) | I am using onnx 1.16.2 onnxruntime-gpu 1.18.1 With cuda 12.x and cudnn 9.x, based on the provided version table, that would work. But I am stuck on this error requiring libcublasLt.so.11 wich does not exist in cuda 12.x instalation. you can pip install onnxruntime-gpu==1.19.0 to work with cuda 12.x and cudnn 9.x. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[Sayyam-Jain]

I'm receiving the same error. Although, when using 1.19, I'm receiving
Segmentation fault (core dumped) and no additional error lines to move forward

[LaurentGoderre]

@Sayyam-Jain It seems to segfault if you don't have Cuda installed.

[yeonsun-ahn]

I'm having the same problem, but is there an exact solution?
Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

Versions of my libraries:

• cuda 12.1
• cudnn 8.9.0
• onnx 1.16.2
• onnxruntime-gpu 1.18.0 (install with external url)
• torch 2.3.1+cu121

[Sayyam-Jain]

I solved it by using:
pip install --no-cache-dir onnxruntime-gpu==1.17.1 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple

[yeonsun-ahn]

@Sayyam-Jain
Thank you for your quick response! I'm replying late because I couldn't check my email.

I solved it by using:
pip install --no-cache-dir onnxruntime-gpu==1.17.1 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple

Still not working for me. Extra index url doesn't seem to apply and cuda 11 version of onnxruntime-gpu 1.17.1 is downloading.
I solved it by uploading my environment to cudnn 9.x and installing onnxruntime-gpu==1.19. Thank you!

[tianleiwu]

For CUDA and cuDNN version, you can look up here:
https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements

Then you can install according to this:
https://onnxruntime.ai/docs/install/#install-onnx-runtime-ort-1

Summary of recommended combinations:

CUDA	cuDNN	PyTorch	onnxruntime-gpu	Install command
12.x	9.x	2.4.1	1.19.2	pip install onnxruntime-gpu==1.19.2
12.x	8.9	2.3.1	1.18.0	pip install onnxruntime-gpu==1.18.0 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
11.8	8.9	2.3.1	1.19.2	pip install onnxruntime-gpu==1.19.2 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/

[NguyenQuocViet42]

I fixed it by just import torch before import onnxruntime:

import torch
import onnxruntime