# Tensors for busy people

### Juan Carlos γ»6 min read

# Wat?

- Whats Scalar?.
- Whats Vector?.
- Whats Matrix?.
- Whats Tensor?.

*This is a quick and short intro to Tensors for busy people...*

*I will keep it KISS and to the minimum possible for easy reading.*

Scalar is the minimum possible unit we are working with, like an integer.

Vector is a bunch of Scalars in a particular axis, can be drawn as a line of boxes or items, lets say is a list or array collection of items.

Matrix is a 2 dimensions arrays, usually represented as a table.

Tensor is a multi-dimensional array, usually can be represented graphically as a cube.

# Rank & Dimension

- Whats Rank?.
- Whats Dimension?.

You can imagine lists of integers, accommodated on different layouts, on a 3D space, or 2D space, etc thats the Rank.

*How it looks like on Code?.*

# Scalar

```
let myscalar = 42
```

Scalar can be a variety of things, usually numeric values, to keep things simple and easy to understand we will use an integer here, `42`

is our Scalar.

# Vector

```
let myvector = [1, 2, 3]
```

Vector is a collection of items, we continue using integers, it can be seen on the code as an array or list, you can draw it as a Rank 1 Vector.

# Matrix

```
let mymatrix = [
[1, 2, 3],
[4, 5, 6],
]
```

We continue adding dimensions then we end up with the Matrix, a 2D Tensor, can be simplified on code as a list with lists inside.

# Tensor

```
let mytensor = [
[
[1, 2, 3],
[4, 5, 6],
],
[
[7, 8, 9],
[10, 11, 12],
],
]
```

Wow, we reached the crazy cube, a multiple dimensions array of integers,

we need to convert this jam of lists into a `Tensor`

object!.

# Tensor Arraymancer

```
import arraymancer
let mytensor = [
[
[1, 2, 3],
[4, 5, 6],
],
[
[7, 8, 9],
[10, 11, 12],
],
].toTensor
```

Done, congrats you coded your first `Tensor`

!.

# Playing with Tensors

*What can I do with it?.*

```
echo mytensor is Tensor # Is really a Tensor type?
echo sizeOf(mytensor) # Whats the Size of it?
echo mytensor.rank # Whats the Rank?
echo mytensor.shape # Whats the Shape of my Tensor?
echo mytensor.strides # Strides?
echo mytensor.offset # Offset? (if any)
```

# Human readable representation

Print a human readable representation using:

```
echo mytensor
```

This should print on the standard output or REPL:

```
Tensor[int] of shape [2, 2, 3] of type "int" on backend "CPU":
| | 1 2 3 | 7 8 9 |
| | 4 5 6 | 10 11 12|
```

On this basic example I am using `CPU`

as Backend, for compatibility,

yours can also say `CUDA`

or `OpenCL`

or `OpenMP`

(if you have the hardware),

because GPUs are comfy massaging numbers.

# Tensor Slicing

You can read or write a value from the Tensor:

(Comments added just to explain the indexes, but not required)

```
import arraymancer
var mytensor = [
[1, 2, 3, 4, 5] # 0
#0 1 2 3 4
].toTensor
echo mytensor[0, 2] # Read the value "3"
mytensor[0, 2] = 42 # Write the value "42"
```

Remember that stuff declared with `var`

is mutable, `let`

is immutable,

a Tensor can also be Compile-Time pre-computed declaring it with `const`

.

We can overwrite that Scalar value, lets see if we can get the whole Column?.

(Comments added to explain indexes, not required, we will take it away soon)

```
import arraymancer
var mytensor = [
[1, 2, 3, 4, 5], # 0
[6, 7, 8, 9, 10], # 1
[11, 12, 13, 14, 15], # 2
[16, 17, 18, 19, 20], # 3
[21, 22, 23, 24, 25], # 4
# 0 1 2 3 4
].toTensor
echo mytensor[_, 2]
```

The only change is the `_`

on the index to select the whole **Column**.

Lets try get an entire **Row** now, without comments this time...

```
import arraymancer
var mytensor = [
[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
].toTensor
echo mytensor[3, _]
```

I hope was not too complicated, just the `_`

for the **Row**.

Now lets grab just the center without touching the rest of it.

```
import arraymancer
var mytensor = [
[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
].toTensor
echo mytensor[2, 2]
```

# Going 3D

Lets chop off a piece of this pie...

```
import arraymancer
var mytensor = [
[
[ 1, 2, 3, 4, 5], # 1st Matrix.
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
],
[
[26, 27, 28, 29, 30], # 2nd Matrix.
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
],
[
[51, 52, 53, 54, 55], # 3rd Matrix.
[56, 57, 58, 59, 60],
[61, 62, 63, 64, 65],
[66, 67, 68, 69, 70],
[71, 72, 73, 74, 75],
],
[
[76, 77, 78, 79, 80], # 4th Matrix.
[81, 82, 83, 84, 85],
[86, 87, 88, 89, 90],
[91, 92, 93, 94, 95],
[96, 97, 98, 99, 100],
],
[
[101, 102, 103, 104, 105], # 5th Matrix.
[106, 107, 108, 109, 110],
[111, 112, 113, 114, 115],
[116, 117, 118, 119, 120],
[121, 122, 123, 124, 125],
]
].toTensor
echo mytensor[3, _, _]
```

What if we want the corner slice?.

```
import arraymancer
var mytensor = [
[
[ 1, 2, 3, 4, 5], # 1st Matrix.
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
],
[
[26, 27, 28, 29, 30], # 2nd Matrix.
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
],
[
[51, 52, 53, 54, 55], # 3rd Matrix.
[56, 57, 58, 59, 60],
[61, 62, 63, 64, 65],
[66, 67, 68, 69, 70],
[71, 72, 73, 74, 75],
],
[
[76, 77, 78, 79, 80], # 4th Matrix.
[81, 82, 83, 84, 85],
[86, 87, 88, 89, 90],
[91, 92, 93, 94, 95],
[96, 97, 98, 99, 100],
],
[
[101, 102, 103, 104, 105], # 5th Matrix.
[106, 107, 108, 109, 110],
[111, 112, 113, 114, 115],
[116, 117, 118, 119, 120],
[121, 122, 123, 124, 125],
]
].toTensor
echo mytensor[_, 1, 4]
```

A last one slicing...

```
import arraymancer
var mytensor = [
[
[ 1, 2, 3, 4, 5], # 1st Matrix.
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
],
[
[26, 27, 28, 29, 30], # 2nd Matrix.
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
],
[
[51, 52, 53, 54, 55], # 3rd Matrix.
[56, 57, 58, 59, 60],
[61, 62, 63, 64, 65],
[66, 67, 68, 69, 70],
[71, 72, 73, 74, 75],
],
[
[76, 77, 78, 79, 80], # 4th Matrix.
[81, 82, 83, 84, 85],
[86, 87, 88, 89, 90],
[91, 92, 93, 94, 95],
[96, 97, 98, 99, 100],
],
[
[101, 102, 103, 104, 105], # 5th Matrix.
[106, 107, 108, 109, 110],
[111, 112, 113, 114, 115],
[116, 117, 118, 119, 120],
[121, 122, 123, 124, 125],
]
].toTensor
echo mytensor[2, 0, 2]
```

# Tensor Exercises

To reduce repetition lets assume this base Tensor:

```
import arraymancer
var mytensor = [
[1, 2],
[3, 4],
].toTensor
```

# Tensor Addition

```
echo mytensor + mytensor
```

# Tensor Substraction

```
echo mytensor - mytensor
```

- Multiplication is left to the reader as exercise.

# Tensor Iterator

```
# Values only.
for value in mytensor:
echo value
# Coordinates and Values.
for coordinates, value in mytensor:
echo coordinates
echo value
```

- Iteration of
`pairs`

and`items`

is left to the reader as exercise.

# Tensor Broadcasting

```
# Broadcasting with operations are made beginning with a dot "."
echo mytensor .+ mytensor # Addition
echo mytensor .- mytensor # Substraction
```

- Broadcasting other operations is left to the reader as exercise.

# Tensor from list

```
# Tensor from vanilla std lib sequence.
let mytensor = toSeq(1..24).toTensor.reshape(2, 3, 4)
```

- Try reshaping your Tensors to new shapes!

# Why Arraymancer?

- C Speed.
- Python like syntax, Numpy like ergonomics.
- Accelerated with Intel MKL/OpenBLAS/CUDA/OpenCL/OpenMP.
- Dependency free distribution.
- 1 Language for the whole stack from experimental quick prototypes to Performance critical Production.

# This is just the beginning, is not the end

This is barely the building block, but you can learn more:

- https://github.com/mratsim/Arraymancer#4-reasons-why-arraymancer
- https://mratsim.github.io/Arraymancer
- https://github.com/mratsim/Arraymancer#installation

*It really wont take more than a nimble install arraymancer to get started on CPU!.*

π