add local layer api #70600
add local layer api #70600
Conversation
winter-wang
force-pushed
the
develop
branch
2 times, most recently
from
11c28f1 to
82e4348
Compare
|
你的PR提交成功，感谢你对开源项目的贡献! |
| from paddle import nn | ||
|
|
||
| class CustomLayer(LocalLayer): | ||
| def __init__(self, mesh): |
There was a problem hiding this comment.
推荐给用户的使用方式，是不是在CustomLayer里直接传入dist_attr更好一些？
There was a problem hiding this comment.
好的，下一个 pr 里面改为转入 out_dist_attrs
| # doctest: +REQUIRES(env:DISTRIBUTED) | ||
| mesh = dist.ProcessMesh([0, 1], dim_names=["x"]) | ||
| custom_layer = CustomLayer(mesh) | ||
| input_tensor = dist.auto_parallel.api.dtensor_from_local( |
There was a problem hiding this comment.
这个示例中，能不能加一些注释展示input_tensor经过local_layer前后的数据变化，以让用户通过这个例子能更直观地理解loacl视角和global视角操作的不同。
There was a problem hiding this comment.
下一个 pr 里面修改为类似 mask_lm_loss 的例子
| inputs[idx] | ||
| ) | ||
| outputs = Layer.__call__(self, *inputs, **kwargs) | ||
| list_outs = paddle.utils.flatten(outputs) |
There was a problem hiding this comment.
这里是否可以检查一下out_dist_attrs和list_outs的数量是否匹配，这应该是用户使用时很容易犯的一个错误。
There was a problem hiding this comment.
这里是否可以检查一下out_dist_attrs和list_outs的数量是否匹配，这应该是用户使用时很容易犯的一个错误。
下一个 pr 添加
4 participants
PR Category
Auto Parallel
PR Types
New features
Description
Add high level api LocalLayer as a wrapper for local view. （#70525）
Other
Pcard-67164