*Memos:
- My post explains Linear Regression in PyTorch.
- My post explains Batch, Mini-Batch and Stochastic Gradient Descent with DataLoader() in PyTorch.
-
My post explains Batch Gradient Descent without
DataLoader()in PyTorch. - My post explains how to save a model in PyTorch.
- My post explains how to load the saved model which I show in this post in PyTorch.
- My post explains Deep Learning Workflow in PyTorch.
- My post explains how to clone a private repository with FGPAT(Fine-Grained Personal Access Token) from Github.
- My post explains how to clone a private repository with PAT(Personal Access Token) from Github.
- My post explains useful IPython magic commands.
- My repo has models.
Module() can create a model, being its base class as shown below:
*Memos:
-
forward() must be overridden in the subclass of
Module(). -
state_dict() can return the dictionary containing parameters and buffers. *It cannot get the
num3andnum4defined without Parameter(). - load_state_dict() can load model's state_dict() into a model. *Basically, it's used to load a saved model into the currently used model.
-
parameters() can return an iterator over module parameters. *It cannot get the
num3andnum4defined withoutParameter(). -
trainingcan check if it's train mode or eval mode. By default, it's train mode. - train() can set a model train mode.
- eval() can set a model eval mode.
- cpu() can convert all model parameters and buffers to CPU.
- cuda() can convert all model parameters and buffers to CUDA(GPU).
- There are also save() and load(). *My post explains
save()andload().
import torch
from torch import nn
class MyModel(nn.Module):
def __init__(self):
super().__init__()
self.num1 = nn.Parameter(torch.tensor(9.))
self.num2 = nn.Parameter(torch.tensor(7.))
self.num3 = torch.tensor(-2.) # Defined without `Parameter()`
self.num4 = torch.tensor(6.) # Defined without `Parameter()`
self.layer1 = nn.Linear(in_features=4, out_features=5)
self.layer2 = nn.Linear(in_features=5, out_features=2)
self.layer3 = nn.Linear(in_features=2, out_features=3)
self.relu = nn.ReLU()
def forward(self, x): # Must be overridden
x1 = self.layer1(input=x)
x2 = self.relu(input=x1)
x3 = self.layer2(input=x2)
x4 = self.relu(input=x3)
x5 = self.layer3(input=x4)
return x5
my_tensor = torch.tensor([8., -3., 0., 1.])
torch.manual_seed(42)
mymodel = MyModel()
mymodel(x=my_tensor)
# tensor([0.8092, 0.8460, 0.3758], grad_fn=<ViewBackward0>)
mymodel
# MyModel(
# (layer1): Linear(in_features=4, out_features=5, bias=True)
# (layer2): Linear(in_features=5, out_features=2, bias=True)
# (layer3): Linear(in_features=2, out_features=3, bias=True)
# (relu): ReLU()
# )
mymodel.layer2
# Linear(in_features=5, out_features=2, bias=True)
mymodel.state_dict()
# OrderedDict([('num1', tensor(9.)),
# ('num2', tensor(7.)),
# ('layer1.weight',
# tensor([[0.3823, 0.4150, -0.1171, 0.4593],
# [-0.1096, 0.1009, -0.2434, 0.2936],
# [0.4408, -0.3668, 0.4346, 0.0936],
# [0.3694, 0.0677, 0.2411, -0.0706],
# [0.3854, 0.0739, -0.2334, 0.1274]])),
# ('layer1.bias',
# tensor([-0.2304, -0.0586, -0.2031, 0.3317, -0.3947])),
# ('layer2.weight',
# tensor([[-0.2062, -0.1263, -0.2689, 0.0422, -0.4417],
# [0.4039, -0.3799, 0.3453, 0.0744, -0.1452]])),
# ('layer2.bias', tensor([0.2764, 0.0697])),
# ('layer3.weight',
# tensor([[0.5713, 0.0773],
# [-0.2230, 0.1900],
# [-0.1918, 0.2976]])),
# ('layer3.bias', tensor([0.6313, 0.4087, -0.3091]))])
mymodel.load_state_dict(state_dict=mymodel.state_dict())
# <All keys matched successfully>
params = mymodel.parameters()
next(params)
# Parameter containing:
# tensor(9., requires_grad=True)
next(params)
# Parameter containing:
# tensor(7., requires_grad=True)
next(params)
# Parameter containing:
# tensor([[0.3823, 0.4150, -0.1171, 0.4593],
# [-0.1096, 0.1009, -0.2434, 0.2936],
# [0.4408, -0.3668, 0.4346, 0.0936],
# [0.3694, 0.0677, 0.2411, -0.0706],
# [0.3854, 0.0739, -0.2334, 0.1274]], requires_grad=True)
next(params)
# Parameter containing:
# tensor([-0.2304, -0.0586, -0.2031, 0.3317, -0.3947],
# requires_grad=True)
next(params)
# Parameter containing:
# tensor([[-0.2062, -0.1263, -0.2689, 0.0422, -0.4417],
# [0.4039, -0.3799, 0.3453, 0.0744, -0.1452]],
# requires_grad=True)
next(params)
# Parameter containing:
# tensor([0.2764, 0.0697], requires_grad=True)
next(params)
# Parameter containing:
# tensor([[0.5713, 0.0773],
# [-0.2230, 0.1900],
# [-0.1918, 0.2976]], requires_grad=True)
next(params)
# Parameter containing:
# tensor([0.6313, 0.4087, -0.3091], requires_grad=True)
mymodel.training
# True
mymodel.eval()
mymodel.training
# False
mymodel.train()
mymodel.training
# True
mymodel.cuda(device='cuda:0')
mymodel.layer2.weight.device
# device(type='cuda', index=0)
mymodel.cpu()
mymodel.layer2.weight.device
# device(type='cpu')
Top comments (0)