Whisper output normalization #14535
Whisper output normalization #14535
Conversation
Leili
requested review from
pablo-garay,
ko3n1g,
thomasdhc and
chtruong814
as code owners
Leili
force-pushed
the
whisper_output_normalization
branch
from
e507dd6 to
9bdb326
Compare
Leili
force-pushed
the
whisper_output_normalization
branch
from
9bdb326 to
5335d50
Compare
| @@ -60,6 +72,8 @@ def __init__(self, cfg: DictConfig, trainer: 'Trainer' = None): | |||
| self.whisper_model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3") | |||
| self.whisper_model.eval() | |||
|
|
|||
| self._normalize_whisper_transcript = cfg.get('normalize_whisper_transcript', True) | |||
There was a problem hiding this comment.
You define _normalize_whisper_transcript in the MagpieTTSModelOfflinePODataGen class but you don't initialize them in this class. Can you add an initialization to this class?
There was a problem hiding this comment.
Hi @blisc
Can you help me understand this comment? I'm initializing the variable to True, if it's not found in the config
There was a problem hiding this comment.
_normalize_whisper_transcript is being initialized inside the if cfg.get('load_whisper_model', False): and is also being used in a similar if statement in the test_step. It should be fine. @blisc is your concern that we might end up using it elsewhere in the code and we should avoid that?
nemo/collections/tts/models/magpietts_preference_optimization.py
Outdated
Show resolved
Hide resolved
|
[🤖]: Hi @Leili 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully So it might be time to merge this PR or get some approvals I'm just a bot so I'll leave it you what to do next. //cc @pablo-garay @ko3n1g |
Signed-off-by: Jason <[email protected]>
|
Will merge this, but will use #14600 instead as we are updating the dev branches |
Important
The
Update branchbutton must only be pressed in very rare occassions.An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.
What does this PR do ?
Adding whisper output normalization for GRPO
Normalization currently covers following languages: english, spanish, french, german, mandarin
If no language is specified, it uses english by default
Collection: [Note which collection this PR will affect]
TTS
Changelog
Usage
# Add a code snippet demonstrating how to use thisGitHub Actions CI
The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".
Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information