We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent b4b1bdc commit 07e5c55Copy full SHA for 07e5c55
examples/language_model/moe/dygraph/run_moe_pretrain.py
@@ -494,7 +494,7 @@ def do_train(args):
494
group=sharding_group,
495
sync_op=True)
496
# Multi stream operation will be supported later
497
- dist.wait(tensor=p, group=sharding_group, sync_op=True)
+ dist.wait(tensor=p, group=sharding_group, use_calc_stream=True)
498
else:
499
initialize_mp_dp_parameters(model, hcg)
500
0 commit comments