Skip to content

embedding层开启sparse=True后,无法查看该层权重的梯度。 #69992

@leftsl

Description

@leftsl

bug描述 Describe the Bug

bug描述 Describe the Bug

description:paddle.nn.embedding层开启sparse=True后无法查看该层权重的梯度了,显示张量未初始化。但是我使用embedding.weight.gradient()可以勉强显示,但是paddle框架的输出结果提示了“该.gradient()马上就要在未来被移除了,尽快改为使用.grad用于查看参数”。而我使用.grad查看该层权重梯度的话就会提示我下面输出结果所示的内容,也就是张量未初始化。
我该如何查看开启sparse=True时的embedding层权重梯度呢?

Running code:

import paddle
import paddle.nn as nn

embedding=nn.Embedding(3,2,sparse=True)
embedding.weight.set_value(paddle.arange(6).reshape((3,2)).astype('float32'))
data=paddle.to_tensor((0,1))
out=embedding(data)
y=out.sum()*2
y.backward()
print(embedding.weight.grad) #这行还可以改为print(embedding.weight.gradient())

output result:

W1201 12:24:43.693264  4996 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.5, Driver API Version: 12.6, Runtime API Version: 12.3
W1201 12:24:43.694262  4996 gpu_resources.cc:164] device: 0, cuDNN Version: 9.0.
Tensor(Not initialized)
(.venv)

其他补充信息 Additional Supplementary Information

感谢技术大佬花费解决问题我的issue,不胜感激。

其他补充信息 Additional Supplementary Information

No response

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions