Skip to content

请教一个L1正则中的疑惑 #790

@CantBlues

Description

@CantBlues

在A2.16.3(L1正则)Line 177 :
当w为正数时,符号为正,值为1,相当于直接乘以w的值;当w为负数时,符号为负,值为-1,相当于乘以(-w)的值。最后的效果就是乘以w的绝对值。
实在没搞懂这句话的意思,不应该是“相当于加/减lambda的值以作为惩罚吗”。最后的效果是乘以w的绝对值??????
附关于L1反向传播的前后文:

def backward(self, delta_in, idx):
    dZ = delta_in
    m = self.x.shape[1]
    if self.regular == RegularMethod.L2:
        self.weights.dW = (np.dot(dZ, self.x.T) + self.lambd * self.weights.W) / m
    elif self.regular == RegularMethod.L1:
        self.weights.dW = (np.dot(dZ, self.x.T) + self.lambd * np.sign(self.weights.W)) / m
    else:
        self.weights.dW = np.dot(dZ, self.x.T) / m
    # end if
    self.weights.dB = np.sum(dZ, axis=1, keepdims=True) / m
    ......

符号函数的效果如下:

>>> a=np.array([1,-1,2,0])
>>> np.sign(a)
>>> array([ 1, -1,  1,  0])

当w为正数时,符号为正,值为1,相当于直接乘以w的值;当w为负数时,符号为负,值为-1,相当于乘以(-w)的值。最后的效果就是乘以w的绝对值。
望赐教

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions