It is time to dig into some real Machine Learning and Data Science coding stuffs. Today I mainly focused on getting started with the Jupyter Notebook workflow and creating a basic project to understand how it works. Finally search for some data set and then follow the basic principles of Machine Learning on it to generate useful information from it. I will also share the notebook I created. The great thing about Jupyter Notebooks it can be literally organized like a blog post or article along with the interactive code, data and other information.

Working with Jupyter Notebooks

I would like to provide a reference to some cool resources to understand the Jupyter Notebook interface, installation guide and its workflow overview.

• Jupyter Notebook Tutorial Video -
• Installation guideline - (It is recommended to install it using Anaconda toolkit as it comes with a lot of useful tools.)
• Documentation -

Since I am a windows user, I would like to provide a quick to tip:

In windows, open Anaconda Prompt from the start menu, navigate to the directory where you want to create jupyter projects, then run the command jupyter notebook. It will open up the notebook in the browser.

As per the basic steps of Machine Learning and Data Science, we shall be creating the project and create a readable notebook that documents the entire process which can then be shared with anyone.

Basics of Data Science and ML using Netflix Shows project

The basic steps of ML and Data Science are:

• Importing data from some source
• Cleaning up the data to remove any irrelevant data if needed
• Splitting up data into Training Set and Test Set.
• Creating a model or an algorithm or a function
• Checking the output
• Improve and repeat the above steps

We shall explore the first two steps in this basic project

1. Importing Data and manipulation

The first and the most important thing for Machine Learning and Data Science is the data itself. To obtain good meaningful conclusions, we must have good data sets. This input data can be collected in a number of ways - from databases, by scraping websites, public APIs or public shared data sets.

Kaggle is a popular website among Machine Learning and Data Science enthusiasts where tons of publicly shared data sets can be found.

I decided to search for a Netflix Shows data set and found this one from Kaggle - https://www.kaggle.com/shivamb/netflix-shows. It contains the data in a CSV format which will be used for this project. After downloading the file, it can be placed in the root directory of the project. I have named it netflix_titles.csv

Since this data is in a kind of tabular format meaning it is arranged in rows and columns, pandas is a great open-source library to process this kind of data and analyze it. It comes along with the Anaconda toolkit, so it can be used directly in the notebook.

import pandas as pd
data_frame = pd.read_csv('netflix_titles.csv')
data_frame.head(10) # show first 10 results
# prints the data frame in as a table

data_frame.info()
# shows information about column data types

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 6234 entries, 0 to 6233
Data columns (total 12 columns):
 #   Column        Non-Null Count  Dtype 
---  ------        --------------  ----- 
 0   show_id       6234 non-null   int64 
 1   type          6234 non-null   object
 2   title         6234 non-null   object
 3   director      4265 non-null   object
 4   cast          5664 non-null   object
 5   country       5758 non-null   object
 6   date_added    6223 non-null   object
 7   release_year  6234 non-null   int64 
 8   rating        6224 non-null   object
 9   duration      6234 non-null   object
 10  listed_in     6234 non-null   object
 11  description   6234 non-null   object
dtypes: int64(2), object(10)
memory usage: 584.6+ KB

data_frame.shape
# provides information of rows and columns as a tuple

(6234, 12)

data_frame.describe()
# shows some basic description

data_frame['title'].head() # lists a specific column data with first 5 entries (head)

0    Norm of the North: King Sized Adventure
1                 Jandino: Whatever it Takes
2                         Transformers Prime
3           Transformers: Robots in Disguise
4                               #realityhigh
Name: title, dtype: object

# Filtering Data
data_frame[data_frame['country'] == 'India'].head()

# Sorting Data
data_frame.sort_values('release_year', ascending=False).head()

This is a great cheat-sheet for Data Science with Python which lists all the commonly used Pandas methods and properties along with other libraries for data science as well.

2. Cleaning Data

The next step is cleaning up data and removing any kinds of information that is not required for the analysis.
Let's consider an example use case where we want to find which Netflix comedy movies and shows that are suitable for all ages(TV-G rating)

# Let's select the relevant columns for analysis
df_shows = pd.DataFrame(data_frame, columns=['title','rating', 'listed_in'])
# filter comedy shows
df_comedy_shows = df_shows[df_shows['listed_in'].str.contains('Comed')]
df_comedy_shows.head()

# filter shows for all ages
df_all_ages = df_comedy_shows[df_comedy_shows['rating']=='TV-G']
df_all_ages.head()

The Github repository for this notebook can be found here

Resources:

• Great collection of Jupyter notebooks for learning
• Data Science cheatsheet

That's all for today's post. Tomorrow will continue exploring more on the other steps of machine learning and data science and perform a visual analysis of data by building charts and diagrams along with creating machine learning models.

Have a great one!