Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions model_zoo/bert/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ python -m paddle.distributed.launch --gpus "0" run_pretrain.py \
--weight_decay 1e-2 \
--adam_epsilon 1e-6 \
--warmup_steps 10000 \
--num_train_epochs 3 \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

num_train_epochs 参数脚本中未用到,设了会报错,所以删了

--input_dir data/ \
--output_dir pretrained_models/ \
--logging_steps 1 \
Expand All @@ -83,7 +82,6 @@ python -m paddle.distributed.launch --gpus "0" run_pretrain.py \
- `weight_decay` 表示AdamW优化器中使用的weight_decay的系数。
- `adam_epsilon` 表示AdamW优化器中使用的epsilon值。
- `warmup_steps` 表示动态学习率热启的step数。
- `num_train_epochs` 表示训练轮数。
- `input_dir` 表示输入数据的目录,该目录下所有文件名中包含training的文件将被作为训练数据。
- `output_dir` 表示模型的保存目录。
- `logging_steps` 表示日志打印间隔。
Expand Down Expand Up @@ -128,7 +126,6 @@ python -m paddle.distributed.launch --xpus "0" run_pretrain.py \
--weight_decay 1e-2 \
--adam_epsilon 1e-6 \
--warmup_steps 10000 \
--num_train_epochs 3 \
--input_dir data/ \
--output_dir pretrained_models/ \
--logging_steps 1 \
Expand All @@ -146,7 +143,6 @@ python -m paddle.distributed.launch --xpus "0" run_pretrain.py \
- `weight_decay` 表示AdamW优化器中使用的weight_decay的系数。
- `adam_epsilon` 表示AdamW优化器中使用的epsilon值。
- `warmup_steps` 表示动态学习率热启的step数。
- `num_train_epochs` 表示训练轮数。
- `input_dir` 表示输入数据的目录,该目录下所有文件名中包含training的文件将被作为训练数据。
- `output_dir` 表示模型的保存目录。
- `logging_steps` 表示日志打印间隔。
Expand Down
4 changes: 4 additions & 0 deletions paddlenlp/trainer/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,10 @@ def __init__(
if model is None:
raise RuntimeError("`Trainer` requires either a `model` or `model_init` argument")

if self.args.to_static:
model = paddle.jit.to_static(model)
logger.info("Successfully to apply @to_static to the whole model.")

if self.args.should_save or self.args.should_save_model_state:
os.makedirs(self.args.output_dir, exist_ok=True)

Expand Down
4 changes: 4 additions & 0 deletions paddlenlp/trainer/training_args.py
Original file line number Diff line number Diff line change
Expand Up @@ -705,6 +705,10 @@ class TrainingArguments:
default=False,
metadata={"help": "Whether to unify hybrid parallel checkpoint."},
)
to_static: Optional[bool] = field(
default=False,
metadata={"help": "Enable training under @to_static."},
)

def __post_init__(self):
env_local_rank = int(os.environ.get("PADDLE_RANK_IN_NODE", -1))
Expand Down