Deep Learning Workflow in PyTorch

1. Prepare data.
2. Prepare (untrained) model.
3. Train model.
4. Evaluate model.
5. Save model.

1. Prepare data.

1. Get data like images, video, sound, text, etc.
2. Divide the data into the one for training(Train data) and the one for testing(Test data). *Basically, train data is 80% and test data is 20%.

2. Prepare (untrained) model.

1. Select the suitable layers for the data. *There are layers in PyTorch such as Linear(), Conv2d(), MaxPool2d(), etc according to the doc.
2. Select the activation function for the data if necesarry. *There are layers in PyTorch such as ReLU(), Sigmoid(), Softmax(), etc according to the doc.

3. Train model.

1. Select the suitable loss function and optimizer for the data. *Memos:
   • A loss function is the function which can get the difference(gap) between predictions and train data.
   • An optimizer is the function which can minimize the loss between predictions and train data with gradient descent. *Gradient Descent(GD) is the function which can find the minimum(or maximum) gradient(slope) of a function.
   • Loss Function is also called Cost Function or Error Function.
   • There are loss functions in PyTorch such as L1Loss(), MSELoss(), CrossEntropyLoss(), etc according to the doc.
   • There are optimizers in PyTorch such as SGD(), Adam(), Adadelta(), etc according to the doc.
2. Calculate predictions with train data.
3. Calculate the loss between predictions and train data with a loss(cost) function.
4. Zero out the gradients of all tensors every epoch for proper calculation. *The gradients are accumulated in buffers, then they are not overwritten until backward() is called.
5. Do backpropagation. *Backpropagation is the algorithm which can minimize the loss between predictions and train data, working from output layer to input layer.
6. Optimize the model to minimize the loss between predictions and train data with an optimizer.

*Repeat the epoch of 2., 3., 4., 5. and 6. to minimize the loss between predictions and train data.

4. Evaluate model.

1. Calculate predictions with test data.
2. Calculate the loss between predictions and test data with a loss(cost) function.
3. Check the loss with train and test data by text or graph.

5. Save model.

Finally, save the model if the model is the enough quality which you want.