Batch is empty when fine-tuning flan-t5 using LoRA

[MorenoLaQuatra]

System Info

• transformers version: 4.38.2
• Platform: Linux-5.15.0-105-generic-x86_64-with-glibc2.35
• Python version: 3.11.9
• Huggingface_hub version: 0.23.2
• Safetensors version: 0.4.3
• Accelerate version: 0.30.1
• Accelerate config: not found
• PyTorch version (GPU?): 2.3.0+cu121 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@muellerzr @SunMarc

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

The issue is reported here: https://discuss.huggingface.co/t/valueerror-the-batch-received-was-empty-your-model-wont-be-able-to-train-on-it-double-check-that-your-training-dataset-contains-keys-expected-by-the-model-args-kwargs-label-ids-label/20200

Expected behavior

Correctly working fine-tuning with LoRA. Anyway, I thing the bug can be solved adding:

self._signature_columns += list(set(["labels", "input_ids"]))

To the function _set_signature_columns_if_needed - here.

May not be the best way to go tho.

[amyeroberts]

Gentle ping @muellerzr @SunMarc

[MekkCyber]

Hello @MorenoLaQuatra,
Sorry for the delayed response, as you suspected, the error is related to the _set_signature_columns_if_needed function. This function removes all dataset columns that don't match the necessary arguments for the model's forward function, along with the columns "label" and "label_ids." While this approach works well for transformer models, where the forward function explicitly lists the required arguments, it doesn't behave as expected for the LowRankAdapterModel. In this case, the forward signature only includes args and kwargs, so all dataset columns get removed, leaving us with an empty batch, which causes the error.
To avoid the error, you can set remove_unused_columns=False (it's True by default) in the TrainingArguments, and manually clean your dataset.

[MorenoLaQuatra]

Is it wrong to assume LoRA models will actually use the same columns as other models (e.g., input_ids and labels)? I got that there maybe workarounds but having it supported by default maybe a good choice, isn't it?

[MekkCyber]

It's not wrong, but I don't think it's the cleanest approach. Setting remove_unused_columns to False seems like a cleaner solution for now, while in future versions we can think about allowing users to override the signature columns themselves if needed

[MorenoLaQuatra]

Got it! Thanks @MekkCyber 👍