cannot import name 'shard_checkpoint' from 'transformers.modeling_utils' (/root/miniconda3/envs/xinference/lib/python3.11/site-packages/transformers/modeling_utils.py) #3920
Description
System Info / 系統信息
Package Version
absl-py 2.1.0
accelerate 0.33.0
aiofiles 23.2.1
aiohappyeyeballs 2.3.4
aiohttp 3.10.1
aioprometheus 23.12.0
aiosignal 1.3.1
alembic 1.13.2
aliyun-python-sdk-core 2.15.1
aliyun-python-sdk-kms 2.16.4
altair 5.4.0
annotated-types 0.7.0
anthropic 0.34.1
antlr4-python3-runtime 4.9.3
anyio 4.4.0
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
async-lru 2.0.4
async-timeout 4.0.3
attrdict 2.0.1
attrs 24.1.0
audioread 3.0.1
auto_gptq 0.7.1
autoawq 0.2.5
autoawq_kernels 0.0.6
autopage 0.5.2
av 12.3.0
babel 2.16.0
bcrypt 3.2.2
beautifulsoup4 4.12.3
bibtexparser 2.0.0b7
bitsandbytes 0.43.3
bleach 6.1.0
boto3 1.28.64
botocore 1.31.85
causal-conv1d 1.4.0
cdifflib 1.2.6
certifi 2019.11.28
cffi 1.17.0
cfgv 3.4.0
chardet 3.0.4
charset-normalizer 3.3.2
chattts 0.1.1
click 8.1.7
cliff 4.7.0
clldutils 3.22.2
cloudpickle 3.0.0
cmaes 0.11.1
cmake 3.30.2
cmd2 2.4.3
colorama 0.4.6
coloredlogs 15.0.1
colorlog 6.8.2
comm 0.2.2
conformer 0.3.2
contourpy 1.2.1
controlnet-aux 0.0.7
crcmod 1.7
cryptography 43.0.0
csvw 3.3.0
ctranslate2 4.3.1
cycler 0.12.1
Cython 3.0.11
datasets 2.20.0
dbus-python 1.2.16
debugpy 1.8.5
decorator 5.1.1
defusedxml 0.7.1
diffusers 0.30.1
dill 0.3.8
diskcache 5.6.3
distlib 0.3.8
distro 1.9.0
distro-info 0.23+ubuntu1.1
dlinfo 1.2.1
ecdsa 0.19.0
editdistance 0.8.1
einops 0.8.0
einx 0.3.0
encodec 0.1.1
eva-decord 0.6.1
exceptiongroup 1.2.2
executing 2.0.1
fastapi 0.110.3
faster-whisper 1.0.3
fastjsonschema 2.20.0
ffmpeg-python 0.2.0
ffmpy 0.4.0
filelock 3.15.4
FlagEmbedding 1.2.11
flashinfer 0.1.5+cu124torch2.4
flatbuffers 24.3.25
fonttools 4.53.1
fqdn 1.5.1
frozendict 2.4.4
frozenlist 1.4.1
fsspec 2024.5.0
funasr 1.1.6
future 1.0.0
fvcore 0.1.5.post20221221
gdown 5.2.0
gekko 1.2.1
gradio 4.26.0
gradio_client 0.15.1
greenlet 3.0.3
grpcio 1.66.0
h11 0.14.0
hf_transfer 0.1.8
hf-xet 1.1.7
hiredis 3.0.0
httpcore 1.0.5
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.34.3
humanfriendly 10.0
hydra-colorlog 1.2.0
hydra-core 1.3.2
hydra-optuna-sweeper 1.2.0
HyperPyYAML 1.2.2
identify 2.6.0
idna 2.8
imageio 2.35.1
imageio-ffmpeg 0.5.1
importlib_metadata 8.4.0
importlib_resources 6.4.4
inflect 7.3.1
iniconfig 2.0.0
interegular 0.3.3
iopath 0.1.10
ipykernel 6.29.5
ipython 8.26.0
ipywidgets 8.1.5
isodate 0.6.1
isoduration 20.11.0
jaconv 0.4.0
jamo 0.4.1
jedi 0.19.1
jieba 0.42.1
Jinja2 3.1.4
jiter 0.5.0
jj-pytorchvideo 0.1.5
jmespath 0.10.0
joblib 1.4.2
json5 0.9.25
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2023.12.1
jupyter_client 8.6.2
jupyter_core 5.7.2
jupyter-events 0.10.0
jupyter-lsp 2.2.5
jupyter_server 2.14.2
jupyter_server_terminals 0.5.3
jupyterlab 4.2.4
jupyterlab_pygments 0.3.0
jupyterlab_server 2.27.3
jupyterlab_widgets 3.0.13
kaldiio 2.18.0
kiwisolver 1.4.5
language-tags 1.2.0
lark 1.1.9
lazy_loader 0.4
libnacl 2.1.0
librosa 0.10.2.post1
lightning 2.4.0
lightning-utilities 0.11.6
litellm 1.43.1
llama_cpp_python 0.2.88
llvmlite 0.43.0
lm-format-enforcer 0.10.3
loguru 0.7.2
loralib 0.1.2
lxml 5.3.0
Mako 1.3.5
mamba-ssm 2.2.2
Markdown 3.7
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matcha-tts 0.0.7.0
matplotlib 3.9.2
matplotlib-inline 0.1.7
mdurl 0.1.2
mistune 3.0.2
modelscope 1.28.1
more-itertools 10.4.0
mpmath 1.3.0
msgpack 1.0.8
multidict 6.0.5
multiprocess 0.70.16
narwhals 1.5.5
natsort 8.4.0
nbclient 0.10.0
nbconvert 7.16.4
nbformat 5.10.4
nemo_text_processing 1.1.0
nest-asyncio 1.6.0
networkx 3.3
ninja 1.11.1.1
nodeenv 1.9.1
notebook 7.2.1
notebook_shim 0.2.4
numba 0.60.0
numpy 1.26.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-ml-py 12.555.43
nvidia-nccl-cu12 2.20.5
nvidia-nvjitlink-cu12 12.6.20
nvidia-nvtx-cu12 12.1.105
omegaconf 2.3.0
onnxruntime 1.19.0
onnxruntime-gpu 1.16.0
openai 1.98.0
openai-whisper 20230306
OpenCC 1.1.6
opencv-contrib-python-headless 4.10.0.84
opencv-python 4.10.0.84
optimum 1.21.4
optuna 2.10.1
orjson 3.10.7
oss2 2.18.6
outlines 0.0.46
overrides 7.7.0
packaging 24.1
pandas 2.2.2
pandocfilters 1.5.1
parameterized 0.9.0
parso 0.8.4
passlib 1.7.4
pbr 6.0.0
peft 0.16.0
pexpect 4.9.0
phonemizer 3.3.0
pillow 10.4.0
pip 24.2
platformdirs 4.2.2
pluggy 1.5.0
plumbum 1.8.3
pooch 1.8.2
portalocker 2.10.1
pre-commit 3.8.0
prettytable 3.11.0
prometheus_client 0.20.0
prometheus-fastapi-instrumentator 7.0.0
prompt_toolkit 3.0.47
protobuf 3.20.2
psutil 6.0.0
ptyprocess 0.7.0
pure_eval 0.2.3
py-cpuinfo 9.0.0
pyairports 2.1.1
pyarrow 17.0.0
pyarrow-hotfix 0.6
pyasn1 0.6.0
pybase16384 0.3.7
pycountry 24.6.1
pycparser 2.22
pycryptodome 3.20.0
pydantic 2.8.2
pydantic_core 2.20.1
pydub 0.25.1
Pygments 2.18.0
PyGObject 3.36.0
pylatexenc 2.10
pynini 2.1.6.post1
pynndescent 0.5.13
pynvml 12.0.0
pyparsing 3.1.3
pyperclip 1.9.0
PySocks 1.7.1
pytest 8.3.2
python-apt 2.0.1+ubuntu0.20.4.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-jose 3.3.0
python-json-logger 2.0.7
python-multipart 0.0.9
pytorch-lightning 2.4.0
pytorch-wpe 0.0.1
pytz 2024.1
PyYAML 6.0.1
pyzmq 26.1.0
quantile-python 1.1
ray 2.34.0
rdflib 7.0.0
redis 5.0.8
referencing 0.35.1
regex 2024.7.24
requests 2.32.3
requests-unixsocket 0.2.0
rfc3339-validator 0.1.4
rfc3986 1.5.0
rfc3986-validator 0.1.1
rich 13.7.1
rootutils 1.0.7
rouge 1.0.1
rpds-py 0.19.1
rpyc 6.0.0
rsa 4.9
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.8
ruff 0.6.2
s3transfer 0.7.0
sacremoses 0.1.1
safetensors 0.4.4
scikit-image 0.24.0
scikit-learn 1.5.1
scipy 1.14.1
seaborn 0.13.2
segments 2.2.1
semantic-version 2.10.0
Send2Trash 1.8.3
sentence-transformers 4.1.0
sentencepiece 0.2.0
setproctitle 1.3.6
setuptools 73.0.1
sglang 0.2.13
shellingham 1.5.4
six 1.14.0
sniffio 1.3.1
soundfile 0.12.1
soupsieve 2.6
soxr 0.4.0
SQLAlchemy 2.0.32
sse-starlette 2.1.3
stack-data 0.6.3
starlette 0.37.2
stevedore 5.3.0
sympy 1.13.1
tabulate 0.9.0
tblib 3.0.0
tensorboard 2.17.1
tensorboard-data-server 0.7.2
tensorboardX 2.6.2.2
tensorizer 2.9.0
termcolor 2.4.0
terminado 0.18.1
threadpoolctl 3.5.0
tifffile 2024.8.10
tiktoken 0.7.0
timm 1.0.9
tinycss2 1.3.0
tokenizers 0.21.4
tomli 2.0.1
tomlkit 0.12.0
torch 2.4.0
torch-complex 0.4.4
torchaudio 2.4.0
torchmetrics 1.4.1
torchvision 0.19.0
tornado 6.4.1
tqdm 4.66.5
traitlets 5.14.3
transformers 4.53.0
transformers-stream-generator 0.0.5
triton 3.0.0
typeguard 4.3.0
typer 0.11.1
types-python-dateutil 2.9.0.20240821
typing_extensions 4.12.2
tzdata 2024.1
umap-learn 0.5.6
unattended-upgrades 0.1
Unidecode 1.3.8
uri-template 1.3.0
uritemplate 4.1.1
urllib3 2.0.7
uvicorn 0.30.5
uvloop 0.19.0
vector-quantize-pytorch 1.16.1
virtualenv 20.26.3
vllm 0.5.4+cu124
vllm-flash-attn 2.6.1
vocos 0.1.0
watchfiles 0.22.0
wcwidth 0.2.13
webcolors 24.8.0
webencodings 0.5.1
websocket-client 1.8.0
websockets 11.0.3
Werkzeug 3.0.4
WeTextProcessing 0.1.10
wget 3.2
wheel 0.34.2
widgetsnbextension 4.0.13
wrapt 1.16.0
xformers 0.0.27.post2
xinference 1.8.1
xoscar 0.7.13
xxhash 3.4.1
yacs 0.1.8
yarl 1.9.4
zipp 3.20.0
zmq 0.0.0
zstandard 0.23.0
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
- docker / docker
- pip install / 通过 pip install 安装
- installation from source / 从源码安装
Version info / 版本信息
1.8.1
The command used to start Xinference / 用以启动 xinference 的命令
docker run -d -v /data:/models -e XINFERENCE_HOME=/models -p 9997:9997 --gpus all registry.cn-hangzhou.aliyuncs.com/xprobe_xinference/xinference:latest xinference-local -H 0.0.0.0 --auth-config /models/model/config.json
Reproduction / 复现过程
加载lora情况下：
降级transformers到4.46不报错，但不支持qwen3。升级transformers则报错cannot import name 'shard_checkpoint' from 'transformers.modeling_utils' (/root/miniconda3/envs/xinference/lib/python3.11/site-packages/transformers/modeling_utils.py)
不加载lora 升级到新版transformers无异常
Expected behavior / 期待表现
期望能正常加载lora