Skip to content

[Bugfix] Pass trust_remote_code_model=True for deepseek examples #1012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 23, 2024

Conversation

dsikka
Copy link
Collaborator

@dsikka dsikka commented Dec 23, 2024

SUMMARY:

  • Pass trust_remote_code_model=True for deepseek models
  • Needed due to the slight differences in how tokenizer and processor pull down their relevant configs

Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

@horheynm
Copy link
Contributor

horheynm commented Dec 23, 2024

why do we need this now? what change caused this?

Needed due to the slight differences in how tokenizer and processor pull down their relevant configs

Sorry missed this

@kylesayrs
Copy link
Collaborator

@horheynm Both AutoConfig and tokenizers share the same cache for AutoConfig, meaning that if you pass trust_remote_code to the processor, you do not have the pass it into oneshot (specifically, the instantiation of AutoConfig in oneshot).

However, the AutoConfig cache is not shared between AutoConfig and processors. Previously we were getting away with not passing trust_remote_code_model because it was sharing from the tokenizer cache, but now we do since processor and AutoConfig no longer shared from the same cache.

@dsikka dsikka merged commit 384059b into main Dec 23, 2024
6 of 8 checks passed
@dsikka dsikka deleted the fix_examples branch December 23, 2024 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants