bug in new transformers: 'Florence2ForConditionalGeneration' object has no attribute '_supports_sdpa'

[pseudotensor]

System Info

transformers 4.55.0
python 3.10
ubuntu 22

Who can help?

@amyeroberts, @qubvel

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

#!/usr/bin/env python3
"""
Minimal reproduction script for Florence-2 AttributeError: '_supports_sdpa'
"""

import torch
import sys

def main():
    print("Loading Florence-2 model...")
    print(f"PyTorch version: {torch.__version__}")
    print(f"Transformers version: {__import__('transformers').__version__}")
    print(f"Python version: {sys.version}")
    
    try:
        from transformers import AutoConfig
        
        # Load config
        config = AutoConfig.from_pretrained(
            "microsoft/Florence-2-base", 
            trust_remote_code=True
        )
        print(f"Config loaded: {type(config)}")
        
        # Use AutoModelForCausalLM to trigger the error
        from transformers import AutoModelForCausalLM
        
        # This should trigger the _supports_sdpa error during model initialization
        print("Loading model with AutoModelForCausalLM...")
        model = AutoModelForCausalLM.from_pretrained(
            "microsoft/Florence-2-base",
            config=config,
            trust_remote_code=True
        )
        print("Success!")
        
    except AttributeError as e:
        if "_supports_sdpa" in str(e):
            print(f"SUCCESS: Reproduced the _supports_sdpa error!")
            print(f"AttributeError: {e}")
            import traceback
            traceback.print_exc()
            return True
        else:
            print(f"Different AttributeError: {e}")
            import traceback
            traceback.print_exc()
            return False
    except Exception as e:
        print(f"Other error: {e}")
        import traceback
        traceback.print_exc()
        return False

if __name__ == "__main__":
    success = main()
    if success:
        print("\n? Successfully reproduced the _supports_sdpa error!")
    else:
        print("\n? Could not reproduce the _supports_sdpa error")(h2ogpt)

gives:

Loading Florence-2 model...
PyTorch version: 2.6.0+cu124
Transformers version: 4.55.0
Python version: 3.10.18 | packaged by conda-forge | (main, Jun  4 2025, 14:45:41) [GCC 13.3.0]
Config loaded: <class 'transformers_modules.microsoft.Florence-2-base.5ca5edf5bd017b9919c05d08aebef5e4c7ac3bac.configuration_florence2.Florence2Config'>
Loading model with AutoModelForCausalLM...
SUCCESS: Reproduced the _supports_sdpa error!
AttributeError: 'Florence2ForConditionalGeneration' object has no attribute '_supports_sdpa'
Traceback (most recent call last):
  File "/home/jon/h2ogpt_internal/florence2_minimal_repro.py", line 30, in main
    model = AutoModelForCausalLM.from_pretrained(
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 593, in from_pretrained
    return model_class.from_pretrained(
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 316, in _wrapper
    return func(*args, **kwargs)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4986, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/jon/.cache/huggingface/modules/transformers_modules/microsoft/Florence-2-base/5ca5edf5bd017b9919c05d08aebef5e4c7ac3bac/modeling_florence2.py", line 2535, in __init__
    super().__init__(config)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2227, in __init__
    self.config._attn_implementation_internal = self._check_and_adjust_attn_implementation(
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2758, in _check_and_adjust_attn_implementation
    return self.get_correct_attn_implementation(applicable_attn_implementation, is_init_check)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2763, in get_correct_attn_implementation
    if not self._supports_sdpa:
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1928, in __getattr__
    raise AttributeError(
AttributeError: 'Florence2ForConditionalGeneration' object has no attribute '_supports_sdpa'

? Successfully reproduced the _supports_sdpa error!

Expected behavior

Shouldn't fail, unsure which transformers it started on.

[pseudotensor]

Seems the breaking change was introduced in transformers 4.54.0

[robin-ede]

This appears to be related to the Florence2 model's custom implementation rather than the core transformers library. The Florence2 model uses trust_remote_code=True and loads custom model code from the Hugging Face Hub instead of using built-in transformers classes.

The custom Florence2ForConditionalGeneration class is missing the _supports_sdpa attribute that the transformers library now expects all model classes to have. This is likely because the model's custom code hasn't been updated to include the newer attribute requirements.

[pseudotensor]

Ok. Seems like a breaking change like this is easy to support migration for.

The chance that microsoft will fix it seems small compared to how easy it is to support migration.

[zucchini-nlp]

True, nudging Microsoft to fix it might take a while. We can wait a bit until Florence 2 is shipped to the core library in #38188 😄 It is already in the last stage of review

[sayakpaul]

Maybe there are many custom modeling implementations that are widely used and might not make it to the core library soon. In that case, it could make sense to have a simple check like:

if getattr(self, "_supports_sdpa"):
    ...

[zucchini-nlp]

Wait, on a second look, if Florence2 is using from_pretrained then it can't miss supports_sdpa. All models inheriting from the PreTrainedModel have it by default set to False

I just looked up Florence code in the hub and found that they changes it from class attribute to a property, thus ig the error when we try to check if the class supports SDPA or not before initializing the class

[zucchini-nlp]

Florence 2 was added in core library the last release, closing as resolved!

[sayakpaul]

Excellent news!

Cc: @DN6 I think you will be delighted!

[david-littlefield]

Fix for AttributeError: BartTokenizerFast has no attribute image_token

After updating to transformers 4.56.0, native Florence-2 support requires using the converted models with dedicated classes:

# Old (with trust_remote_code)
from transformers import AutoProcessor, AutoModelForCausalLM

MODEL_ID = "microsoft/Florence-2-large"

self.processor = AutoProcessor.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    ...
)

self.model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    ...
)

The ducviet00 models are identical weights, just converted for native support.

# New (native support)
from transformers import Florence2Processor, Florence2ForConditionalGeneration

MODEL_ID = "ducviet00/Florence-2-large-hf"

self.processor = Florence2Processor.from_pretrained(
    MODEL_ID,
    ...
)

self.model = Florence2ForConditionalGeneration.from_pretrained(
    MODEL_ID,
    ...
)

[fcakyon]

Thanks for the native implementation! @ducviet00 @zucchini-nlp @sayakpaul @david-littlefield.

The finetuned variants of Florence2 models (microsoft/Florence-2-large-ft [30k downloads] and microsoft/Florence-2-base-ft [49k downloads]) perform good on few academic benchmarks (shorter overfitted captions and phrases) but in reality non-finetuned models (microsoft/Florence-2-large [580k downloads] and microsoft/Florence-2-base [820K downloads]) perform better (longer, more diverse, underfitted generations). You can confirm this by their monthly download counts.

Why do you only support these underperforming finetuned variants in the native Florence2 implementation? 🤔

Can you guide me on how I can convert other Florence2 weights to the new supported format? 🙏🏻

[zucchini-nlp]

@fcakyon hey, you can convert the weights by running the models/convert_florence_hf_format.py file and changing the model id there. I believe it would work out of the box since the tuned and pretrained models have the same architecture and format

In the meanwhile, we are also asking Microsoft to put converted weights in their official hub repo. That will help a lot with discoverability and we'll make sure to convert all released checkpoints