Skip to content

Commit ff5f57c

Browse files
committed
[Trainer] Fix release_grads (PaddlePaddle#9085)
* fix pp release_grads
1 parent 33fc0c7 commit ff5f57c

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

paddlenlp/trainer/training_args.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1038,7 +1038,7 @@ def __post_init__(self):
10381038
"dp_comm_overlap": enable_dp_comm_overlap,
10391039
"sharding_comm_overlap": enable_sharding_comm_overlap,
10401040
"enable_timer": "enable_timer" in pipeline_parallel_config,
1041-
"release_gradients": "enable_release_grads" in pipeline_parallel_config,
1041+
"release_gradients": "enable_release_grads" in pipeline_parallel_config or self.release_grads,
10421042
"overlap_p2p_comm": "enable_overlap_p2p_comm" in pipeline_parallel_config,
10431043
"clear_every_step_cache": "enable_clear_every_step_cache" in pipeline_parallel_config,
10441044
"use_batch_p2p_comm": "disable_batch_p2p_comm" not in pipeline_parallel_config,

0 commit comments

Comments
 (0)