add new API: paddle.clone;Tensor.element_size;nn.utils.parameters_to_vector

[zhwesky2010]

PR types

New features

PR changes

APIs

Describe

1.New API: paddle.clone

Returns a copy of input Tensor. It will always have a Tensor copy.

Tn addition, the OP provides gradient propagation.

import paddle

x = paddle.ones([2], stop_gradient=False)
clone_x = paddle.clone(x)

y = clone_x**3
y.backward()
print(clone_x.grad)          # [3]
print(x.grad)                     # [3]

2.New API: paddle.Tensor.element_size

Returns the size in bytes of an element in the Tensor.

import paddle

x = paddle.static.data(name='x', shape=[3, 2], dtype='bool')
x.element_size() # 1

x = paddle.static.data(name='x', shape=[3, 2], dtype='int16')
x.element_size() # 2

x = paddle.static.data(name='x', shape=[3, 2], dtype='float16')
x.element_size() # 2

x = paddle.static.data(name='x', shape=[3, 2], dtype='float32')
x.element_size() # 4

x = paddle.static.data(name='x', shape=[3, 2], dtype='float64')
x.element_size() # 8

3.New API：paddle.nn.utils.parameters_to_vector

Flatten parameters to a 1-D Tensor.

import paddle
linear = paddle.nn.Linear(10, 15)

paddle.nn.utils.parameters_to_vector(linear.parameters())
# 1-D Tensor: [165]

4.New API：paddle.nn.utils.vector_to_parameters

Transform a Tensor with 1-D shape to the parameters.

import paddle
linear1 = paddle.nn.Linear(10, 15, paddle.ParamAttr(paddle.nn.initializer.Constant(3.)))

vec = paddle.nn.utils.parameters_to_vector(linear1.parameters())

linear2 = paddle.nn.Linear(10, 15)
# copy weight of linear1 to linear2
paddle.nn.utils.vector_to_parameters(vec, linear2.parameters())  
# weight: Tensor(shape=[10, 15], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
#                 [[3. , ..., 3. ],
#                  [..., ..., ...],
#                  [3. , ..., 3. ]])

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[TCChenlong]

这两句描述不够清楚，可以在改一下

[zhwesky2010]

Done

[TCChenlong]

Tn addition -> In addition

[zhwesky2010]

Done, thx

[TCChenlong]

LGTM

[Xreki]

能否用1个split代替多个slice？

[zhwesky2010]

Done，已去掉多个slice

[Xreki]

为什么这里需要显式使用inplace接口呢？

[zhwesky2010]

core.ops.reshape2_是inplace机制，core.ops.reshape2是view机制。使用view机制的OP是写死的，目前有4个：

inplace机制下：输入、输出是同一个var。
view机制下：需要创建一个输出var，但输出var会和输入var是share_buffer_with的，所以两者都没有成本。
这里指定用inplace版本的，写法会简单些，不需要创建一个临时输出var

[Xreki]

vector_to_parameters可能会比较慢

[zhwesky2010]

已去掉assign，改成最简化的写法：

parameters_to_vector：使用1个concat OP + 完全inplace的reshape_ OP
vector_to_parameters：使用1个split OP + 完全inplace的reshape_ OP

目前全部使用inplace版的reshape_ OP，每个API实质上只有1个concat或split OP的耗时，性能可以得到保证了

[TCChenlong]

LGTM

[luotao1]

LGTM for CMakeLists.txt