DEV Community

🙌Top 10 🐍 Python libraries for any ML projects 🚀

Marine on November 13, 2023

TL;DR In this article, I’ll give you the ultimate Python libraries for any Machine Learning project: the must-know libraries for each ...

Read full post

Prayson Wilfred Daniel • Nov 14 '23 • Edited

Awesome! I did not know the first one. My pure ML list:

scikit-learn - Python 🐍 ML
PyMC - Bayesian ML - CausalPy
flaml - fast library for AutoML and tuning.
hummingbird - ML compile models to Tensor
mljar-supervised
modAL - A modular active learning framework for Python
Neuraxle - Code Machine Learning Pipelines - The Right Way.
HyperGBM - AutoML
poniard - AutoML -> Cross Validation?
autogluon - AutoML {Image, Text, Time Series, Tabular Data} with ⌛ limits. Also works with 🤟 imodels
skrub - data preprocessing made easier (automated)

I have not started with time series nor CI/CD in ML 😋

Marine • Nov 16 '23 • Edited

That's a great list, will definitely take time to look into some I don't know like Skrub or poniard. Thanks for sharing!

Jeffrey Ip • Nov 13 '23

Here's a bonus one: Here's a bonus one: github.com/confident-ai/deepeval

Randell Brian Knight • Nov 15 '23

Thanks for providing this awesome list! 🎉

Alexey Yuzhakov • Nov 13 '23

Taipy link points to CatBoost )

Marine • Nov 14 '23

Updated, thank you!

chopslip • Nov 13 '23

This sounds really good, thanks for sharing!

Nathan Tarbert • Nov 14 '23

Nice list! Thanks for sharing

Rym • Nov 13 '23

Hey, thanks Marine for this clear article :)

Nevo David • Nov 13 '23

Great ML list!
Thank you for sharing!

AleaJactaEst • Nov 13 '23

Love it, thank you for your article!

thaddaeustedcode • Nov 14 '23

Python is great

Anne • Nov 13 '23

Great article Marine ! I want to get into machine learning, this is definitely helpful, thxxx 👍🏼🙌🏼

Kevin • Sep 23 • Edited

Hey Marine, this article is great! I love some of the libraries that you mentioned such as Pandas, Numpy, Keras, and Tensorflow. Here are my top 10 (actually 12 because I wanted to include Computer Vision libraries) from my perspective:

Python Libraries for Data Manipulation and Analysis

Pandas:
- DataFrames for efficient data manipulation (sorting, filtering, grouping).
- Tools for data cleaning, addressing missing values and inconsistencies.
- Simple data filtering and conditional queries.
- Merging and joining datasets for comprehensive analysis.
Numpy:
- Efficient handling of arrays and numerical operations.
- Vectorization allows for operations on entire arrays without loops.
- Built-in functions for complex linear algebra tasks.

Visualization Libraries

Matplotlib:
- Versatile plotting options from line charts to heatmaps.
- Extensive customization for detailed control over plot elements.
- Manages figures and subplots for intricate data relationships.
Seaborn:
- Enhances visual appeal with built-in themes and color palettes.
- Simplifies statistical visualization, making charts more aesthetically pleasing.

Traditional Machine Learning Tools

Scikit-Learn (Sklearn):
- Preprocessing techniques to prepare data for ML models.
- Diverse set of algorithms like decision trees and ensemble methods.
- Built-in tools for model evaluation and comparison.
XGBoost:
- Powerful gradient boosting algorithm for high performance.
- Regularization techniques to prevent overfitting.
- Efficient handling of missing values during training.

Natural Language Processing (NLP) Libraries

NLTK:
- Tokenization and parsing for text segmentation.
- Corpus resources for training NLP models.
- Part-of-speech tagging for grammatical analysis.
Gensim:
- Specializes in topic modeling, word embedding, and document similarity.
- Handles large-scale text processing with efficiency.
- Provides pre-trained models and datasets for NLP tasks.
Transformers (Hugging Face):
- State-of-the-art pre-trained models like BERT and GPT.
- Fine-tuning capabilities for customizing models.
- Multilingual support for global NLP projects.

Deep Learning Libraries

TensorFlow:
- Highly scalable, working across multiple CPUs and GPUs.
- Flexible architecture for complex neural networks.
- Visualization tools like Tensorboard for tracking model training.
PyTorch:
- Dynamic computational graph for flexible neural network development.
- Intuitive, Pythonic syntax for ease of use.
- Tensor operations that mimic Numpy for efficient data handling.
Keras:
- High-level API for easy neural network design and prototyping.
- Modular approach for flexible model architecture.
- Works with multiple backends like TensorFlow, Theano, and CNTK.

Computer Vision Libraries

OpenCV:
- Image processing tools for enhancing and transforming images.
- Object detection for recognizing objects in images and videos.
- Feature matching for aligning image features.
Dlib:
- Face recognition and shape analysis tools.
- Expression analysis for detecting emotional states in faces.
- Shape prediction for detailed object analysis, especially faces and hands.

This covers the core features of each library for AI and ML development! Also, along with the article I mentioned by my colleague Nicolas Azevedo: Python Libraries for Machine Learning, I also recommend this article: Hugging Face, which is focused 100% on the Transformers (Hugging Face) library.