*Memos:
- My post explains ELU, SELU and CELU.
- My post explains heaviside() and Identity().
- My post explains ReLU() and LeakyReLU().
- My post explains PReLU() and ELU().
- My post explains GELU() and Mish().
- My post explains SiLU() and Softplus().
- My post explains Tanh() and Softsign().
- My post explains Sigmoid() and Softmax().
SELU() can get the 0D or more D tensor of the zero or more values computed by SELU function from the 0D or more D tensor of zero or more elements as shown below:
*Memos:
- The 1st argument for initialization is
inplace(Optional-Default:False-Type:bool): *Memos:- It does in-place operation.
- Keep it
Falsebecause it's problematic withTrue.
- The 1st argument is
input(Required-Type:tensoroffloat).
import torch
from torch import nn
my_tensor = torch.tensor([8., -3., 0., 1., 5., -2., -1., 4.])
selu = nn.SELU()
selu(input=my_tensor)
# tensor([8.4056, -1.6706, 0.0000, 1.0507, 5.2535, -1.5202, -1.1113, 4.2028])
selu
# SELU()
selu.inplace
# False
selu = nn.SELU(inplace=True)
selu(input=my_tensor)
# tensor([8.4056, -1.6706, 0.0000, 1.0507, 5.2535, -1.5202, -1.1113, 4.2028])
my_tensor = torch.tensor([[8., -3., 0., 1.],
[5., -2., -1., 4.]])
selu = nn.SELU()
selu(input=my_tensor)
# tensor([[8.4056, -1.6706, 0.0000, 1.0507],
# [5.2535, -1.5202, -1.1113, 4.2028]])
my_tensor = torch.tensor([[[8., -3.], [0., 1.]],
[[5., -2.], [-1., 4.]]])
selu = nn.SELU()
selu(input=my_tensor)
# tensor([[[8.4056, -1.6706], [0.0000, 1.0507]],
# [[5.2535, -1.5202], [-1.1113, 4.2028]]])
CELU() can get the 0D or more D tensor of the zero or more values computed by CELU function from the 0D or more D tensor of zero or more elements as shown below:
*Memos:
- The 1st argument for initialization is
alpha(Optional-Default:1.0-Type:float). *It's applied to negative input values. - The 2nd argument for initialization is
inplace(Optional-Default:False-Type:bool): *Memos:- It does in-place operation.
- Keep it
Falsebecause it's problematic withTrue.
- The 1st argument is
input(Required-Type:tensoroffloat).
import torch
from torch import nn
my_tensor = torch.tensor([8., -3., 0., 1., 5., -2., -1., 4.])
celu = nn.CELU()
celu(input=my_tensor)
# tensor([8.0000, -0.9502, 0.0000, 1.0000, 5.0000, -0.8647, -0.6321, 4.0000])
celu
# CELU(alpha=1.0)
celu.alpha
# 1.0
celu.inplace
# False
celu = nn.CELU(alpha=1.0, inplace=True)
celu(input=my_tensor)
# tensor([8.0000, -0.9502, 0.0000, 1.0000, 5.0000, -0.8647, -0.6321, 4.0000])
my_tensor = torch.tensor([[8., -3., 0., 1.],
[5., -2., -1., 4.]])
celu = nn.CELU()
celu(input=my_tensor)
# tensor([[8.0000, -0.9502, 0.0000, 1.0000],
# [5.0000, -0.8647, -0.6321, 4.0000]])
my_tensor = torch.tensor([[[8., -3.], [0., 1.]],
[[5., -2.], [-1., 4.]]])
celu = nn.CELU()
celu(input=my_tensor)
# tensor([[[8.0000, -0.9502], [0.0000, 1.0000]],
# [[5.0000, -0.8647], [-0.6321, 4.0000]]])
Top comments (0)