Have you ever wondered how YouTube, Netflix shows you video/movie recommendations?

Credit: Pradip Nichte

One of the techniques used to build a recommendations system is called collaborative filtering.

The idea behind collaborative filtering is that if User A and User B have watched some movies in common, they have similar preferences, and User A is more likely to watch films seen by user B than any random user preferences.

But how do we find similar users for a given user?

We can represent users and movies using a matrix (or spreadsheet).
Each row represents each user, and each column represents each movie.

Each cell in the matrix is 1 or 0, which tells us whether the user has watched a movie or not.

One way to calculate the similarity between two users (rows) is using dot product or cosine similarity.

Let's pick user A; once we compare the user A row with every row, we can find the most similar row; let's call it User X.

Then pick a movie from User X which A has not watched yet and Show it to user A.

Can we improve the above technique?

1. Instead of finding one similar user, we can see the top 5 similar users and show one movie from each user or recommending a film seen by all five similar users and not watched by User A.

2. Represent each cell in the matrix by rating (1 to 5) given by the user for that movie instead of 1 or 0.

3. Since the system will be having millions of users and 1000s of movies, the matrix will become huge, we can use matrix factorization techniques to represent users and movies efficiently.