*Memos:
- My post explains Step function, Identity and ReLU.
- My post explains heaviside() and Identity().
- My post explains PReLU() and ELU().
- My post explains SELU() and CELU().
- My post explains GELU() and Mish().
- My post explains SiLU() and Softplus().
- My post explains Tanh() and Softsign().
- My post explains Sigmoid() and Softmax().
ReLU() can get the 0D or more D tensor of the zero or more values computed by ReLU function from the 0D or more D tensor of zero or more elements as shown below:
*Memos:
- The 1st argument for initialization is
inplace(Optional-Default:False-Type:bool): *Memos:- It does in-place operation.
- Keep it
Falsebecause it's problematic withTrue.
- The 1st argument is
input(Required-Type:tensorofintorfloat). - You can also use relu() with a tensor.
import torch
from torch import nn
my_tensor = torch.tensor([8, -3, 0, 1, 5, -2, -1, 4])
relu = nn.ReLU()
relu(input=my_tensor)
my_tensor.relu()
# tensor([8, 0, 0, 1, 5, 0, 0, 4])
relu
# ReLU()
relu.inplace
# False
relu = nn.ReLU(inplace=True)
relu(input=my_tensor)
# tensor([8, 0, 0, 1, 5, 0, 0, 4])
my_tensor = torch.tensor([[8, -3, 0, 1],
[5, 0, -1, 4]])
relu = nn.ReLU()
relu(input=my_tensor)
# tensor([[8, 0, 0, 1],
# [5, 0, 0, 4]])
my_tensor = torch.tensor([[[8, -3], [0, 1]],
[[5, 0], [-1, 4]]])
relu = nn.ReLU()
relu(input=my_tensor)
# tensor([[[8, 0], [0, 1]],
# [[5, 0], [0, 4]]])
my_tensor = torch.tensor([[[8., -3.], [0., 1.]],
[[5., 0.], [-1., 4.]]])
relu = nn.ReLU()
relu(input=my_tensor)
# tensor([[[8., 0.], [0., 1.]],
# [[5., 0.], [0., 4.]]])
LeakyReLU() can get the 0D or more D tensor of the zero or more values computed by LeakyReLU function from the 0D or more D tensor of zero or more elements as shown below:
*Memos:
- The 1st argument for initialization is
negative_slope(Optional-Default:0.01-Type:float). *It's applied to negative input values. - The 2nd argument for initialization is
inplace(Optional-Default:False-Type:bool): *Memos:- It does in-place operation.
- Keep it
Falsebecause it's problematic withTrue.
- The 1st argument is
input(Required-Type:tensoroffloat).
import torch
from torch import nn
my_tensor = torch.tensor([8., -3., 0., 1., 5., -2., -1., 4.])
lrelu = nn.LeakyReLU()
lrelu(input=my_tensor)
# tensor([8.0000, -0.0300, 0.0000, 1.0000, 5.0000, -0.0200, -0.0100, 4.0000])
lrelu
# LeakyReLU(negative_slope=0.01)
lrelu.negative_slope
# 0.01
lrelu.inplace
# False
lrelu = nn.LeakyReLU(negative_slope=0.01, inplace=True)
lrelu(input=my_tensor)
# tensor([8.0000, -0.0300, 0.0000, 1.0000, 5.0000, -0.0200, -0.0100, 4.0000])
my_tensor = torch.tensor([[8., -3., 0., 1.],
[5., -2., -1., 4.]])
lrelu = nn.LeakyReLU()
lrelu(input=my_tensor)
# tensor([[8.0000, -0.0300, 0.0000, 1.0000],
# [5.0000, -0.0200, -0.0100, 4.0000]])
my_tensor = torch.tensor([[[8., -3.], [0., 1.]],
[[5., -2.], [-1., 4.]]])
lrelu = nn.LeakyReLU()
lrelu(input=my_tensor)
# tensor([[[8.0000, -0.0300], [0.0000, 1.0000]],
# [[5.0000, -0.0200], [-0.0100, 4.0000]]])
Top comments (0)