ImportError: cannot import name 'hf_bucket_url' from 'transformers.file_utils'

[angus-lherrou]

On a fresh install of the package, cnlpt.train_system cannot be instantiated due to an error:

>>> import cnlpt.train_system
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/angus/miniconda3/envs/cnlpt-fresh-39/lib/python3.9/site-packages/cnlpt/train_system.py", line 47, in <module>
    from transformers.file_utils import hf_bucket_url, CONFIG_NAME
ImportError: cannot import name 'hf_bucket_url' from 'transformers.file_utils' (/home/angus/miniconda3/envs/cnlpt-fresh-39/lib/python3.9/site-packages/transformers/file_utils.py)

[angus-lherrou]

I'm guessing this has to do with an updated dependency. Investigating now.

@mikix do you get this error too if you try a fresh install on Python 3.9?

[angus-lherrou]

I've isolated the error to transformers version 4.22. The error occurs with 4.22.0 but not with 4.21.3.

[angus-lherrou]

The commit that breaks this is huggingface/transformers@377cdde. As per huggingface/transformers#18497, many "internal utilities" were removed without notice (🙄🤬🫠), and the functionality is ostensibly available in the huggingface-hub package instead. Investigating now...