-
Notifications
You must be signed in to change notification settings - Fork 3k
Open
Description
在A2.16.3(L1正则)Line 177 :
当w为正数时,符号为正,值为1,相当于直接乘以w的值;当w为负数时,符号为负,值为-1,相当于乘以(-w)的值。最后的效果就是乘以w的绝对值。
实在没搞懂这句话的意思,不应该是“相当于加/减lambda的值以作为惩罚吗”。最后的效果是乘以w的绝对值??????
附关于L1反向传播的前后文:
def backward(self, delta_in, idx):
dZ = delta_in
m = self.x.shape[1]
if self.regular == RegularMethod.L2:
self.weights.dW = (np.dot(dZ, self.x.T) + self.lambd * self.weights.W) / m
elif self.regular == RegularMethod.L1:
self.weights.dW = (np.dot(dZ, self.x.T) + self.lambd * np.sign(self.weights.W)) / m
else:
self.weights.dW = np.dot(dZ, self.x.T) / m
# end if
self.weights.dB = np.sum(dZ, axis=1, keepdims=True) / m
......
符号函数的效果如下:
>>> a=np.array([1,-1,2,0])
>>> np.sign(a)
>>> array([ 1, -1, 1, 0])
当w为正数时,符号为正,值为1,相当于直接乘以w的值;当w为负数时,符号为负,值为-1,相当于乘以(-w)的值。最后的效果就是乘以w的绝对值。
望赐教
Metadata
Metadata
Assignees
Labels
No labels