Categorization of Machine Learning Algorithms

Throughout our academic or tech career, we have encountered different AI and ML algorithms. Terms such as Supervised Learning, Reinforcement Learning, K-Means algorithm, K-Nearest Neighbor algorithm, DBSCAN algorithm, etc. repeatedly show up. However, we often fail to organize all these terms into a cohesive framework.

This article organizes and categorizes most of the widely used machine learning algorithms.

Artificial Intelligence (AI) has six subsets:

1. Machine Learning
2. Natural Language Processing
3. Deep Learning
4. Robotics
5. Speech Recognition
6. Expert Systems

Machine Learning (ML)

• it is one of the subsets of AI
• consists of 4 types

1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning
4. Semi-Supervised learning

Each of these learning types use various algorithms and they are listed below.

1. Supervised Learning

• used in Classification and Regression problems.

Under Classification domain, some popularly used algorithms are:

i. Logistic regression algorithm
ii. Naive Bayes algorithm
iii. K-Nearest Neighbor algorithm
iv. Support Vector Machine
v. Decision Tree
vi. Random Forest
vii. Gradient Boosting Machines (XGBoost, LightGBM)
viii. Neural Network algorithms

Under Regression domain, some popularly used algorithms are:

i. Linear Regression
ii. Polynomial Regression
iii. Support Vector Machine
iv. Decision Tree
v. Random Forest
vi. Gradient boosting

Note: Some algorithms like Decision tree, Random forest, Support Vector and Gradient boosting can be used for both classification and regression problems.

2. Unsupervised Learning

• used in Clustering, Dimensionality Reduction and Association Rule Learning problems.

Under Clustering domain, some popular algorithms are:

i. K-Means algorithm
ii. Hierarchal clustering
iii. DBSCAN algorithm
iv. Gaussian Mixture Models
v. Mean Shift

Under Dimensionality Reduction domain, some popular algorithms are:

i. Principal Component Analysis (PCA)
ii. Autoencoders
iii. tSNE algorithm
iv. UMAP algorithm

Under Association Rule Learning domain, some popular algorithms are:

i. Apriori algorithm
ii. FPGrowth algorithm

3. Semi-Supervised Learning

• some of the popular algorithms under this learning are:

i. Self-training algorithm
ii. Mixture Models
iii. Graph based methods
iv. Transductive support vector machines

4. Reinforcement Learning

• consists of four methods:

a. Valued based method
b. Policy based method
c. Model based method
d. Actor-Critic method

a. Value based method

• consists of following algorithms:

i. Q Learning
ii. Deep Q-Networks
iii. Double DQN
iv. Dueling DQN

b. Policy based method

• consists of following algorithms:

i. Reinforce algorithm
ii. Deterministic policy gradient

c. Model based method

• consists of following algorithms:

i. Model predictive control
ii. Monte Carlo tree search

d. Actor-Critic method

• consists of following algorithms:

i. Advantage actor-critic (A2C)
ii. Asynchronous advantage actor-critic (A3C)
iii. Soft actor-critic (SAC)

Visualizing the AI/ML terminologies in a broader picture helps to better understand the algorithms and its use cases.