Skip to content

Conversation

@rasbt
Copy link
Owner

@rasbt rasbt commented Oct 17, 2025

This adds an easier toggle to the Qwen3 dense notebooks to switch between the models more intuitively

New text:

  • Note that there are two models, the "base" and the "hybrid" model, and the hybrid model can be used as either a reasoning or a regular instruction-following model:
  • In short, the model types are as follows:
    • base: the pretrained base model; note that the Qwen3 pretraining contained some reasoning data (chain-of-thought data), so the model sometimes emits reasoning traces even though it didn't undergo the reasoning training (reinforcement learning) stages
    • hybrid
      • reasoning: emits long reasoning traces inside <think></think> tags
      • instruct: the same as above, but long reasoning traces can be suppressed by manually adding empty <think></think> (this is done by the tokenizer); this way, the model acts like a regular instruction-following model
# Select which model to use via the following flag; only one can be True

USE_BASE_MODEL = False
USE_REASONING_MODEL = True 
USE_INSTRUCT_MODEL = False

if (USE_BASE_MODEL + USE_REASONING_MODEL
    + USE_INSTRUCT_MODEL) != 1:
    raise AttributeError("Only one of the options above can be True.")

Fixes #836

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@rasbt rasbt merged commit 27b6dfa into main Oct 17, 2025
13 checks passed
@rasbt rasbt deleted the instruct-use branch October 17, 2025 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Probably incorrect usage of USE_REASONING_MODEL and USE_INSTRUCT_MODEL flags for Qwen3Tokenizer

2 participants