# Homemade Machine Learning in Python

### Oleksii Trekhleb γ»9 min read

*Image source*

Iβve recently launched **Homemade Machine Learning** repository that contains examples of popular machine learning algorithms and approaches (like *linear/logistic regressions, K-Means clustering, neural networks*) implemented in **Python** with mathematics behind them being explained. Each algorithm has interactive **Jupyter Notebook** demo that allows you to play with training data, algorithms configurations and immediately see the results, charts and predictions **right in your browser**. In most cases the explanations are based on this great machine learning course by Andrew Ng.

The purpose of the repository was *not* to implement machine learning algorithms by using 3rd party library βone-linersβ *but *rather to practice implementing these algorithms from scratch and get better understanding of the mathematics behind each algorithm. Thatβs why all algorithms implementations are called βhomemadeβ.

The main Python libraries that are used there are NumPy and Pandas. These two are used for efficient matrix operations and for loading/parsing CSV datasets. When it comes to Jupyter Notebook demos then such libraries as Matplotlib and Plotly are being used for data visualizations.

Currently the following topics have been covered:

#### Regression: Linear Regression

In regression problems we do real value predictions. Basically we try to draw a line/plane/n-dimensional plane along the training examples.

*Usage examples: stock price forecast, sales analysis, dependency of any number, etc.*

- π Linear Regression Mathβββtheory and links for further readings
- βοΈ Linear Regression Implementation Example
- βΆοΈ Demo | Univariate Linear Regressionβββpredict
`country happiness`

score by`economy GDP`

- βΆοΈ Demo | Multivariate Linear Regressionβββpredict
`country happiness`

score by`economy GDP`

and`freedom index`

- βΆοΈ Demo | Non-linear Regressionβββuse linear regression with
*polynomial*and*sinusoid*features to predict non-linear dependencies.

#### Classification: Logistic Regression

In classification problems we split input examples by certain characteristic.

*Usage examples: spam-filters, language detection, finding similar documents, handwritten letters recognition, etc.*

- π Logistic Regression Mathβββtheory and links for further readings
- βοΈ Logistic Regression Implementation Example
- βΆοΈ Demo | Logistic Regression (Linear Boundary)βββpredict Iris flower
`class`

based on`petal_length`

and`petal_width`

- βΆοΈ Demo | Logistic Regression (Non-Linear Boundary)βββpredict microchip
`validity`

based on`param_1`

and`param_2`

- βΆοΈ Demo | Multivariate Logistic Regressionβββrecognize handwritten digits from
`28x28`

pixel images.

#### Clustering: K-means Algorithm

In clustering problems we split the training examples by unknown characteristics. The algorithm itself decides what characteristic to use for splitting.

*Usage examples: market segmentation, social networks analysis, organize computing clusters, astronomical data analysis, image compression, etc.*

- π K-means Algorithm Mathβββtheory and links for further readings
- βοΈ K-means Algorithm Implementation Example
- βΆοΈ Demo | K-means Algorithmβββsplit Iris flowers into clusters based on
`petal_length`

and`petal_width`

#### Neural Networks: Multilayer Perceptron (MLP)

The neural network itself isnβt an algorithm, but rather a framework for many different machine learning algorithms to work together and process complex data inputs.

*Usage examples: as a substitute of all other algorithms in general, image recognition, voice recognition, image processing (applying specific style), language translation, etc.*

- π Multilayer Perceptron Mathβββtheory and links for further readings
- βοΈ Multilayer Perceptron Implementation Example
- βΆοΈ Demo | Multilayer Perceptronβββrecognize handwritten digits from
`28x28`

pixel images.

#### Anomaly Detection: Gaussian Distribution

Anomaly detection (also outlier detection) is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data.

*Usage examples: intrusion detection, fraud detection, system health monitoring, removing anomalous data from the dataset etc.*

βββββββββββββββββ

I hope youβll find the repository useful. Either by playing with demos or by reading the math sections or by simply exploring the source code. Happy coding!

Learn machine learning from scratch is absolutely delightful . I personally have learned implementation of linear and logistic regression using Matlab and Python . Also , it is really fun to explore neural networks and the math behind them especially backward propagation.

This was really an amazing article to read .

Great repo ! I'm trying to learn ML in my spare time

Also checkout fast.ai

Good luck with the learning, Emmanuel!

Useful Post!...please make more such posts with study and example links for other ML topics as well.β₯οΈ

For Xmas, my wife has gotten me

Introduction to Machine Learning with Python, by Mueller and Guido.Time to stick my toes in the water. ;-) I hope that's a reasonably good introductory book.

That's a good Christmas gift! :) Good luck with studying!

Good job!

Thank you!