In this post, you will start from the very basics of deep learning frameworks. TENSORS
So basically a neural network is the bunch of nodes that take one or more input, processes them and returns one or more output.
Table of Content
- What are Tensors
- Creating Tensors
- Tensor Types and Their Conversions
- Shapes and Dimensions
Tensors are the building blocks of a neural network. A PyTorch tensor is the generalized form of arrays in dimensions to run arbitrary computations on GPU. [READ MORE]
A tensor can be of any dimension. In this course you will learn about the following dimensions only
- 0D -> Scalar numbers like
20etc. are 0-Dimensional tensors
- 1D -> Array of numbers like
[1, 2, 3]or
[20, 30, 10]etc.
- 2D -> Matrix of numbers like
[[1, 2, 3], [2, 3, 4]]
Note During the process of learning, the network applies some tensor methods on these tensor values in order to gain the desired output.
Before creating and using tensors in PyTorch, install it by following the instructions from here: https://pytorch.org/get-started/locally/
You can create the tensors by importing
torch and using
import torch t0 = torch.tensor(10) # 0-D tensor t1 = torch.tensor([1, 2, 3, 4]) # 1-D tensor t2 = torch.tensor([[1, 2, 3], [2, 3, 4]]) # 2-D tensor print(t0) print(t1) print(t2)
tensor(10) tensor([1, 2, 3, 4]) tensor([[1, 2, 3], [2, 3, 4]])
To convert it into numpy array, simply use
Tensor.numpy() method and use it as you wish
array([[1, 2, 3], [2, 3, 4]])
PyTorch supports the following data-types of the tensors
Note Difference between
.type() is better explained here
To explicitly create a tensor of specific type, either pass
dtype while defining the tensor in
torch.tensor method or use tensor constructor.
For example, here I am creating a float tensor by providing int32 data
tf0 = torch.tensor([1, 2, 3, 4], dtype=torch.float32) print(tf0) print(tf0.dtype) tf1 = torch.FloatTensor([5, 6, 7, 8]) print(tf1) print(tf1.dtype)
tensor([1., 2., 3., 4.]) torch.float32 tensor([5., 6., 7., 8.]) torch.float32
You can pass-in the the datatype in the
Tensor.type([type]) method, in order to change the data-type of the existing tensor.
For example, changing float tensor to int32
tf0 = tf0.type(torch.int32) # after changing, it returns a new tensor object of that type print(tf0.dtype)
While training the model, you will deal with higher dimensions and neural network only accept the dimension which is defined at input layer while architecting the model.
The dimension basically tells whether the tensor is 0-D or 1-D or 2-D or even higher than that. Dimension of tensor is also called the rank of the tensor.
print(t1.size()) print(t1.ndim) print(t2.shape) print(t2.ndimension())
torch.Size() 1 torch.Size([2, 3]) 2
Sometimes working with neural network, you will face a situation when you are supposed to change the dimension of the tensor. This can be done by using
Tensor.view(nrows, ncols) method of the tensor
For example, I have a tensor
[[1, 2, 3], [2, 3, 4]] and I want to convert it to 2D with 1 row and 6 columns
tx = t2.view(1, 6) print(tx)
tensor([[1, 2, 3, 2, 3, 4]])
Note The number of parameters in
.view method is the total number of dimensions
# Converting 2D to 1D tensor tx = t2.view(6) print(tx)
tensor([1, 2, 3, 2, 3, 4])
There are special values for the view method. Suppose you don't know how many rows would be good while resizing your matrix, you can set it to -1 it will automatically adjust the rows for you. Same goes with columns.
# Telling PyTorch to keep the max 2 elements in one column and make as many rows as you want tr = t2.view(-1, 2) print(tr) # Telling PyTorch to keep max 2 rows and adjust columns automatically tc = t2.view(2, -1) tc
tensor([[1, 2], [3, 2], [3, 4]]) tensor([[1, 2, 3], [2, 3, 4]])
Note Both the rows and columns can't be set to
-1 at same time.
I hope you have liked this post. Please share it with your friends and colleagues and help them learn the concepts of PyTorch. If you have doubts or any idea reach me out via following sources