DEV Community

Juan Carlos
Juan Carlos

Posted on

Tensors for busy people

#tensors #machinelearning #beginners #hpc

Wat?

  • Whats Scalar?.
  • Whats Vector?.
  • Whats Matrix?.
  • Whats Tensor?.

This is a quick and short intro to Tensors for busy people...
I will keep it KISS and to the minimum possible for easy reading.

Scalar is the minimum possible unit we are working with, like an integer.

Vector is a bunch of Scalars in a particular axis, can be drawn as a line of boxes or items, lets say is a list or array collection of items.

Matrix is a 2 dimensions arrays, usually represented as a table.

Tensor is a multi-dimensional array, usually can be represented graphically as a cube.

Rank & Dimension

  • Whats Rank?.
  • Whats Dimension?.

You can imagine lists of integers, accommodated on different layouts, on a 3D space, or 2D space, etc thats the Rank.

How it looks like on Code?.

Scalar 

let myscalar = 42
Enter fullscreen mode Exit fullscreen mode

Scalar can be a variety of things, usually numeric values, to keep things simple and easy to understand we will use an integer here, 42 is our Scalar.

Vector 

let myvector = [1, 2, 3]
Enter fullscreen mode Exit fullscreen mode

Vector is a collection of items, we continue using integers, it can be seen on the code as an array or list, you can draw it as a Rank 1 Vector.

Matrix 

let mymatrix = [
                 [1, 2, 3],
                 [4, 5, 6],
               ]
Enter fullscreen mode Exit fullscreen mode

We continue adding dimensions then we end up with the Matrix, a 2D Tensor, can be simplified on code as a list with lists inside.

Tensor 

let mytensor = [
                 [
                   [1, 2, 3],
                   [4, 5, 6],
                 ],
                 [
                   [7,   8,  9],
                   [10, 11, 12],
                 ],
               ]
Enter fullscreen mode Exit fullscreen mode

Wow, we reached the crazy cube, a multiple dimensions array of integers,
we need to convert this jam of lists into a Tensor object!.

Tensor Arraymancer 

import arraymancer

let mytensor = [
                 [
                   [1, 2, 3],
                   [4, 5, 6],
                 ],
                 [
                   [7,   8,  9],
                   [10, 11, 12],
                 ],
               ].toTensor
Enter fullscreen mode Exit fullscreen mode

Done, congrats you coded your first Tensor!.

Playing with Tensors

What can I do with it?.

echo mytensor is Tensor  # Is really a Tensor type?
echo sizeOf(mytensor)    # Whats the Size of it?
echo mytensor.rank       # Whats the Rank? 
echo mytensor.shape      # Whats the Shape of my Tensor?
echo mytensor.strides    # Strides?
echo mytensor.offset     # Offset? (if any)
Enter fullscreen mode Exit fullscreen mode

Human readable representation

Print a human readable representation using:

echo mytensor
Enter fullscreen mode Exit fullscreen mode

This should print on the standard output or REPL:

Tensor[int] of shape [2, 2, 3] of type "int" on backend "CPU":
| |     1       2       3 |     7       8       9 |
| |     4       5       6 |     10      11      12|
Enter fullscreen mode Exit fullscreen mode

On this basic example I am using CPU as Backend, for compatibility,
yours can also say CUDA or OpenCL or OpenMP (if you have the hardware),
because GPUs are comfy massaging numbers.

Tensor Slicing

You can read or write a value from the Tensor:

(Comments added just to explain the indexes, but not required)


import arraymancer

var mytensor = [
    [1, 2, 3, 4, 5] # 0
    #0  1  2  3  4
  ].toTensor

echo mytensor[0, 2]  # Read the value "3"
mytensor[0, 2] = 42  # Write the value "42"
Enter fullscreen mode Exit fullscreen mode

Remember that stuff declared with var is mutable, let is immutable,
a Tensor can also be Compile-Time pre-computed declaring it with const.

We can overwrite that Scalar value, lets see if we can get the whole Column?.

(Comments added to explain indexes, not required, we will take it away soon)


import arraymancer

var mytensor = [
    [1,   2,  3,  4,  5],   # 0
    [6,   7,  8,  9, 10],   # 1
    [11, 12, 13, 14, 15],   # 2
    [16, 17, 18, 19, 20],   # 3
    [21, 22, 23, 24, 25],   # 4
    # 0   1   2   3   4
  ].toTensor

echo mytensor[_, 2]
Enter fullscreen mode Exit fullscreen mode

The only change is the _ on the index to select the whole Column.

Lets try get an entire Row now, without comments this time...


import arraymancer

var mytensor = [
    [1,   2,  3,  4,  5], 
    [6,   7,  8,  9, 10],  
    [11, 12, 13, 14, 15],  
    [16, 17, 18, 19, 20], 
    [21, 22, 23, 24, 25],  
  ].toTensor

echo mytensor[3, _]
Enter fullscreen mode Exit fullscreen mode

I hope was not too complicated, just the _ for the Row.

Now lets grab just the center without touching the rest of it.


import arraymancer

var mytensor = [
    [1,   2,  3,  4,  5],  
    [6,   7,  8,  9, 10],  
    [11, 12, 13, 14, 15],  
    [16, 17, 18, 19, 20],  
    [21, 22, 23, 24, 25],  
  ].toTensor

echo mytensor[2, 2]
Enter fullscreen mode Exit fullscreen mode

Going 3D

Lets chop off a piece of this pie...


import arraymancer

var mytensor = [
    [
      [ 1,  2,  3,  4,  5],      # 1st Matrix.
      [ 6,  7,  8,  9, 10],
      [11, 12, 13, 14, 15],
      [16, 17, 18, 19, 20],
      [21, 22, 23, 24, 25],
    ],
    [
      [26, 27, 28, 29, 30],      # 2nd Matrix.
      [31, 32, 33, 34, 35],
      [36, 37, 38, 39, 40],
      [41, 42, 43, 44, 45],
      [46, 47, 48, 49, 50],
    ],
    [
      [51, 52, 53, 54, 55],      # 3rd Matrix.
      [56, 57, 58, 59, 60],
      [61, 62, 63, 64, 65],
      [66, 67, 68, 69, 70],
      [71, 72, 73, 74, 75],
    ],
    [
      [76, 77, 78, 79,  80],     # 4th Matrix.
      [81, 82, 83, 84,  85],
      [86, 87, 88, 89,  90],
      [91, 92, 93, 94,  95],
      [96, 97, 98, 99, 100],
    ],
    [
      [101, 102, 103, 104, 105], # 5th Matrix.
      [106, 107, 108, 109, 110],
      [111, 112, 113, 114, 115],
      [116, 117, 118, 119, 120],
      [121, 122, 123, 124, 125],
    ]
  ].toTensor

echo mytensor[3, _, _]
Enter fullscreen mode Exit fullscreen mode

What if we want the corner slice?.


import arraymancer

var mytensor = [
    [
      [ 1,  2,  3,  4,  5],      # 1st Matrix.
      [ 6,  7,  8,  9, 10],
      [11, 12, 13, 14, 15],
      [16, 17, 18, 19, 20],
      [21, 22, 23, 24, 25],
    ],
    [
      [26, 27, 28, 29, 30],      # 2nd Matrix.
      [31, 32, 33, 34, 35],
      [36, 37, 38, 39, 40],
      [41, 42, 43, 44, 45],
      [46, 47, 48, 49, 50],
    ],
    [
      [51, 52, 53, 54, 55],      # 3rd Matrix.
      [56, 57, 58, 59, 60],
      [61, 62, 63, 64, 65],
      [66, 67, 68, 69, 70],
      [71, 72, 73, 74, 75],
    ],
    [
      [76, 77, 78, 79,  80],     # 4th Matrix.
      [81, 82, 83, 84,  85],
      [86, 87, 88, 89,  90],
      [91, 92, 93, 94,  95],
      [96, 97, 98, 99, 100],
    ],
    [
      [101, 102, 103, 104, 105], # 5th Matrix.
      [106, 107, 108, 109, 110],
      [111, 112, 113, 114, 115],
      [116, 117, 118, 119, 120],
      [121, 122, 123, 124, 125],
    ]
  ].toTensor

echo mytensor[_, 1, 4]
Enter fullscreen mode Exit fullscreen mode

A last one slicing...


import arraymancer

var mytensor = [
    [
      [ 1,  2,  3,  4,  5],      # 1st Matrix.
      [ 6,  7,  8,  9, 10],
      [11, 12, 13, 14, 15],
      [16, 17, 18, 19, 20],
      [21, 22, 23, 24, 25],
    ],
    [
      [26, 27, 28, 29, 30],      # 2nd Matrix.
      [31, 32, 33, 34, 35],
      [36, 37, 38, 39, 40],
      [41, 42, 43, 44, 45],
      [46, 47, 48, 49, 50],
    ],
    [
      [51, 52, 53, 54, 55],      # 3rd Matrix.
      [56, 57, 58, 59, 60],
      [61, 62, 63, 64, 65],
      [66, 67, 68, 69, 70],
      [71, 72, 73, 74, 75],
    ],
    [
      [76, 77, 78, 79,  80],     # 4th Matrix.
      [81, 82, 83, 84,  85],
      [86, 87, 88, 89,  90],
      [91, 92, 93, 94,  95],
      [96, 97, 98, 99, 100],
    ],
    [
      [101, 102, 103, 104, 105], # 5th Matrix.
      [106, 107, 108, 109, 110],
      [111, 112, 113, 114, 115],
      [116, 117, 118, 119, 120],
      [121, 122, 123, 124, 125],
    ]
  ].toTensor

echo mytensor[2, 0, 2]
Enter fullscreen mode Exit fullscreen mode

Tensor Exercises

To reduce repetition lets assume this base Tensor:

import arraymancer

var mytensor = [
    [1, 2],
    [3, 4],
  ].toTensor
Enter fullscreen mode Exit fullscreen mode

Tensor Addition 

echo mytensor + mytensor
Enter fullscreen mode Exit fullscreen mode

Tensor Substraction 

echo mytensor - mytensor
Enter fullscreen mode Exit fullscreen mode
  • Multiplication is left to the reader as exercise.

Tensor Iterator 

# Values only.
for value in mytensor:
  echo value   

# Coordinates and Values.
for coordinates, value in mytensor:
  echo coordinates
  echo value
Enter fullscreen mode Exit fullscreen mode
  • Iteration of pairs and items is left to the reader as exercise.

Tensor Broadcasting 

# Broadcasting with operations are made beginning with a dot "."
echo mytensor .+ mytensor    # Addition
echo mytensor .- mytensor    # Substraction
Enter fullscreen mode Exit fullscreen mode
  • Broadcasting other operations is left to the reader as exercise.

Tensor from list 

# Tensor from vanilla std lib sequence.
let mytensor = toSeq(1..24).toTensor.reshape(2, 3, 4)
Enter fullscreen mode Exit fullscreen mode
  • Try reshaping your Tensors to new shapes!

Why Arraymancer?

  • C Speed.
  • Python like syntax, Numpy like ergonomics.
  • Accelerated with Intel MKL/OpenBLAS/CUDA/OpenCL/OpenMP.
  • Dependency free distribution.
  • 1 Language for the whole stack from experimental quick prototypes to Performance critical Production.

This is just the beginning, is not the end

This is barely the building block, but you can learn more:

It really wont take more than a nimble install arraymancer to get started on CPU!.
👑

Top comments (0)

Read next

valeriavg profile image

Day 9: Terminal Forms 📇

Valeria -

shanewen profile image

New to Dev.to. What do you usually do here?

Shane Wen -

code42cate profile image

Day 17: Github Actions

Jonas Scholz -

jfelipegarcia profile image

While Loops

Juan Garcia -