🏗️ Refactor top-entropy in GRPO

[qgallouedec]

What does this PR do?

Some minor change, late review from #3563

1. Don't use/unittest private method: _compute_entropy_mask -> get_high_entropy_mask
2. token_entropy_percentile_threshold -> top_entropy_quantile top better align with the top-20% message (its more a quantile than a percentile here.
3. Fix the doc: link and var name (rho, not tau)
4. I better like _get_per_token_logps_and_entropies returning a tuple, but personal preference

cc @pramodith happy to have you thoughts on this

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[shirinyamani]

Im wondering the default being 1 has better result? cuz it seems like this would mask all but the highest entropy token?
i think cuz actually top_entropy_quantile = max(entropies of non-padding tokens)
So this way only the tokens with exactly the highest entropy will be marked True ?

[qgallouedec]

Im wondering the default being 1 has better result? cuz it seems like this would mask all but the highest entropy token?

Actually it's the opposite, I fixed the doc here:
5621e1a

[pramodith]

Thanks for the changes! I'll take a look later today.

[pramodith]

The test case corresponding to training a model with the entropy mask needs to be updated to use the new argument top_entropy_quantile for the test to pass.