Skip to content

[Bug] Unable to connect to Hugging Face's MultiPLE-Evaluator #2229

@qingting-zhxy

Description

@qingting-zhxy

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

{'CUDA available': True,
'CUDA_HOME': '/usr/local/cuda-12.4',
'GCC': 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0',
'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H100 80GB HBM3',
'MMEngine': '0.10.7',
'MUSA available': False,
'NVCC': 'Cuda compilation tools, release 12.4, V12.4.131',
'OpenCV': '4.11.0',
'PyTorch': '2.5.0',
'PyTorch compiling details': 'PyTorch built with:\n'
' - GCC 11.4\n'
' - C++ Version: 201703\n'
' - Intel(R) MKL-DNN v3.5.3 (Git Hash '
'66f0cb9eb66affd2da3bf5f8d897376f04aae6af)\n'
' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
' - NNPACK is enabled\n'
' - CPU capability usage: AVX512\n'
' - CUDA Runtime 12.4\n'
' - NVCC architecture flags: '
'-gencode;arch=compute_90,code=sm_90\n'
' - CuDNN 90.1\n'
' - Build settings: BUILD_TYPE=Release, '
'CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, '
'CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= '
'-D_GLIBCXX_USE_CXX11_ABI=1 '
'-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
'-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER '
'-DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM '
'-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK '
'-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC '
'-Wall -Wextra -Werror=return-type '
'-Werror=non-virtual-dtor '
'-Werror=range-loop-construct '
'-Werror=bool-operation -Wnarrowing '
'-Wno-missing-field-initializers '
'-Wno-type-limits -Wno-array-bounds '
'-Wno-unknown-pragmas -Wno-unused-parameter '
'-Wno-strict-overflow -Wno-strict-aliasing '
'-Wno-stringop-overflow -Wsuggest-override '
'-Wno-psabi -Wno-error=old-style-cast '
'-Wno-missing-braces -fdiagnostics-color=always '
'-faligned-new -Wno-unused-but-set-variable '
'-Wno-maybe-uninitialized -fno-math-errno '
'-fno-trapping-math -Werror=format '
'-Wno-stringop-overflow, PERF_WITH_AVX=1, '
'PERF_WITH_AVX2=1, TORCH_VERSION=2.5.0, '
'USE_CUDA=1, USE_CUDNN=ON, USE_CUSPARSELT=OFF, '
'USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, '
'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, '
'USE_MKL=OFF, USE_MKLDNN=ON, USE_MPI=OFF, '
'USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, '
'USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, \n',
'Python': '3.10.18 (main, Jun 5 2025, 13:14:17) [GCC 11.2.0]',
'lmdeploy': "not installed:No module named 'lmdeploy'",
'numpy_random_seed': 2147483648,
'opencompass': '0.4.2+',
'sys.platform': 'linux',
'transformers': '4.51.3'}

Reproduces the problem - code/configuration sample

from gradio_client import Client

ip_address='https://opencompass-multiple-evaluator.hf.space'
client = Client(ip_address)

Reproduces the problem - command or script

None

Reproduces the problem - error message

Loaded as API: https://opencompass-multiple-evaluator.hf.space/
Traceback (most recent call last):
File "/home/workspace/test.py", line 4, in
client = Client(ip_address)
File "/root/miniconda/envs/torch/lib/python3.10/site-packages/gradio_client/client.py", line 153, in init
self.config = self._get_config()
File "/root/miniconda/envs/torch/lib/python3.10/site-packages/gradio_client/client.py", line 890, in _get_config
return r.json()
File "/root/miniconda/envs/torch/lib/python3.10/site-packages/httpx/_models.py", line 766, in json
return jsonlib.loads(self.content, **kwargs)
File "/root/miniconda/envs/torch/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/root/miniconda/envs/torch/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/root/miniconda/envs/torch/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Other information

The MultiPLE-Evaluator Space on Hugging Face is stuck at "Preparing Space" and cannot be accessed

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions