[Model] use AutoWeightsLoader for bart

[calvin0327]

FIX (partial) #15697

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[calvin0327]

@lengrongfu PTAL

[lengrongfu]

Thanks for working on this, there are some comments.

[calvin0327]

test:

[calvin0327]

@DarkLight1337 @noooop I've resolved it, and BART can now be successfully loaded.

[noooop]

Is BartModel.load_weight not used and can be deleted? Because BartForConditionalGeneration already uses AutoWeightsLoader.

or

Can the BartModel.load_weights logic be simplified using AutoWeightsLoader?

then

BartForConditionalGeneration.load_weights:

    def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]):
        return self.model.load_weights(weights, mapper=self.hf_to_vllm_mapper)

---

The final result of AutoWeightsLoader replaces each model's load_weights function.

Now the load_weights logic still not simplified.

[calvin0327]

Is BartModel.load_weight not used and can be deleted? Because BartForConditionalGeneration already uses AutoWeightsLoader.

or

Can the BartModel.load_weights logic be simplified using AutoWeightsLoader?

then

BartForConditionalGeneration.load_weights:

    def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]):
        return self.model.load_weights(weights, mapper=self.hf_to_vllm_mapper)

The final result of AutoWeightsLoader replaces each model's load_weights function.

Now the load_weights logic still not simplified.

@noooop Now,BartModel.load_weights is used, it be called by BartForConditionalGeneration.load_weights. I've noticed that other models also add the load_weights method to their base models, which is then invoked via AutoWeightsLoader.

Please correct me if I'm misunderstanding.

[noooop]

You have optimized the logic related to lm_head and tie_word_embeddings.

Try using AutoWeightsLoader to simplify the BartModel.load_weights logic.

[calvin0327]

@noooop Thank you for your reply, but I haven't found a way to optimize the BartModel.load_weight, could you give me some hints?

[noooop]

Please refer to #20534

[calvin0327]

This is test result:

[calvin0327]

@noooop Thanks for your hints, I had optimized the load_weights for BartModel, PTAL

[noooop]

@noooop Thanks for your hints, I had optimized the load_weights for BartModel, PTAL

Please see the message above.

[noooop]

thanks for your contribution.

cc @DarkLight1337

[noooop]

v1/entrypoints/openai/test_multi_api_servers.py::test_single_completion[ibm-research/PowerMoE-3b] Testing locally

I think it's failure not related to this PR's , but it has failed twice already.

---

can pass locally, I don't know why it failed.

[noooop]

@calvin0327

I found another failure related to ibm-research/PowerMoE-3b, so it's failure not related to this PR's

https://buildkite.com/vllm/ci/builds/24219/steps/canvas?sid=019818b9-c1bf-4856-909d-04ed3afc8696

Try tomorrow, click on calvin0327:autoweights-for-bart under the title -> sync fork -> update branch to merge into main and trigger CI. Also, auto-merge will not be canceled. Do not submit new code, it will cancel auto-merge.

[calvin0327]

@calvin0327

I found another failure related to ibm-research/PowerMoE-3b, so it's failure not related to this PR's

https://buildkite.com/vllm/ci/builds/24219/steps/canvas?sid=019818b9-c1bf-4856-909d-04ed3afc8696

Try tomorrow, click on calvin0327:autoweights-for-bart under the title -> sync fork -> update branch to merge into main and trigger CI. Also, auto-merge will not be canceled. Do not submit new code, it will cancel auto-merge.

Ok. thanks.