Improve HF device handling

[rmitsch]

Description

Improve HF device handling:

• Consider device/device_map conflicts.
• Always move inputs to model device for models using AutoModelForCausalLM.

Context/motivation: #324 (reply in thread), #324. This PR should help with both.

Corresponding documentation PR

-

Types of change

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran all tests in tests and usage_examples/tests, and all new and existing tests passed. This includes
  • all external tests (i. e. pytest ran with --external)
  • all tests requiring a GPU (i. e. pytest ran with --gpu)
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[adrianeboyd]

I do think it's a good idea to test with accelerate, at least in the future. It looks like that torch bug will be fixed in torch v2.1.1.

[rmitsch]

I do think it's a good idea to test with accelerate, at least in the future. It looks like that torch bug will be fixed in torch v2.1.1.

I'll do that in a follow-up PR as I want this to be included in the upcoming v0.6.3 release (release today or tomorrow).

[svlandeg]

There's no documentation update necessary after this?

[rmitsch]

There's no documentation update necessary after this?

I wouldn't think so - this is part bugfix and part the ability to process torch_dtype in the config, which should have been there anyway, as we claim all HF args can be set and are passed on to the HF model. I'd say describing that in the release notes should be sufficient, but we can make this explicit in the docs as well.