DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Big data models 📊 vs. Computer memory 💾
marisogo profile

Big data models 📊 vs. Computer memory 💾

#bigdata #pipeline #dataengineering #dask
186
Comments 3
11 min read
Machine learning 101
ulianaev profile

Machine learning 101

#machinelearning #datascience #dataengineering
85
Comments
8 min read
A Beginner’s Guide to Building LLM-Powered Applications with LangChain!
pavanbelagatti profile

A Beginner’s Guide to Building LLM-Powered Applications with LangChain!

#datascience #dataengineering #llm #database
54
Comments 6
7 min read
Python Cheat Sheet for Data Engineers and Data Scientists!
pavanbelagatti profile

Python Cheat Sheet for Data Engineers and Data Scientists!

#python #datascience #dataengineering #developers
52
Comments
3 min read
Modern Data Engineering RoadMap - 2024
grayhat profile

Modern Data Engineering RoadMap - 2024

#dataengineering #datascience #analytics #python
33
Comments 3
3 min read
Dummy Variable Trap in Machine Learning
jagroop2001 profile

Dummy Variable Trap in Machine Learning

#machinelearning #deeplearning #dataengineering #datapreprocessing
19
Comments
2 min read
Data Analysis of the Titanic with Python!
pavanbelagatti profile

Data Analysis of the Titanic with Python!

#python #datascience #dataengineering #database
17
Comments 1
6 min read
The Pains of Data Ingestion
burakkarakan profile

The Pains of Data Ingestion

#data #dataengineering #beginners #programming
16
Comments 3
6 min read
How NASCAR delivers realtime racing data to millions of fans around the world
maddysquinn profile
Maddy Quinn for Ably

How NASCAR delivers realtime racing data to millions of fans around the world

#database #dataengineering #webdev #interview
16
Comments
2 min read
[Free E-Book] The World of Vector Databases & AI Applications!
pavanbelagatti profile

[Free E-Book] The World of Vector Databases & AI Applications!

#database #datascience #dataengineering #ai
14
Comments
3 min read
8 Rusty open source data projects to watch in 2024 🤩
debadyuti profile

8 Rusty open source data projects to watch in 2024 🤩

#rust #data #opensource #dataengineering
14
Comments 11
7 min read
How moving from Pandas to Polars made me write better code without writing better code
duvenagep profile

How moving from Pandas to Polars made me write better code without writing better code

#polars #dataengineering #rust #airflow
12
Comments 3
14 min read
The Data Engineering Docker-Compose Starter Kit
olgabraginskaya profile

The Data Engineering Docker-Compose Starter Kit

#docker #tutorial #development #dataengineering
10
Comments
13 min read
Sentiment Analysis Using Python: A Beginner-Friendly Tutorial!
pavanbelagatti profile

Sentiment Analysis Using Python: A Beginner-Friendly Tutorial!

#python #datascience #dataengineering #tutorial
10
Comments
4 min read
How Vector Databases Work: A Hands-On Tutorial!
pavanbelagatti profile

How Vector Databases Work: A Hands-On Tutorial!

#database #ai #datascience #dataengineering
10
Comments
9 min read
Data Engineer vs. Business Intelligence Data Analyst
ai_jobsnet profile

Data Engineer vs. Business Intelligence Data Analyst

#dataengineering #analyst #career #programming
9
Comments
4 min read
Top 10 Common Data Engineers and Scientists Pain Points in 2024
bobur profile

Top 10 Common Data Engineers and Scientists Pain Points in 2024

#datascience #python #dataengineering #data
9
Comments
5 min read
Installing Python Packages in AWS Glue using AWS CodeArtifact
klescosia profile

Installing Python Packages in AWS Glue using AWS CodeArtifact

#tutorial #python #aws #dataengineering
9
Comments
6 min read
A Comprehensive Dive into the New Time-Series Storage Engine - Mito
greptime profile

A Comprehensive Dive into the New Time-Series Storage Engine - Mito

#rust #database #dataengineering #greptime
8
Comments
5 min read
Introduction to Data Science
shaheryaryousaf profile

Introduction to Data Science

#datascience #machinelearning #ai #dataengineering
8
Comments 4
2 min read
Different file formats, a benchmark doing basic operations
pedrohgoncalves profile

Different file formats, a benchmark doing basic operations

#dataengineering #spark #benchmark #datascience
8
Comments 2
9 min read
The easiest way to navigate through MongoDB, PySpark, and Jupyter Notebook
ndurumo254 profile

The easiest way to navigate through MongoDB, PySpark, and Jupyter Notebook

#datascience #dataengineering #mongodb #etl
7
Comments
3 min read
How to Transpose Columns in Each Group to a Single Row
esproc_spl profile
Judy

How to Transpose Columns in Each Group to a Single Row

#sql #database #coding #dataengineering
7
Comments
2 min read
Ultimate Guide: Best Books for Data Science with Ratings for All Levels
imadadrees profile

Ultimate Guide: Best Books for Data Science with Ratings for All Levels

#datascience #books #beginners #dataengineering
7
Comments
8 min read
MLOps project setup: Airbyte - Supabase connection
dplata profile

MLOps project setup: Airbyte - Supabase connection

#mlops #dataengineering
7
Comments
1 min read
Exploratory Data Analysis Using Data Visualization Techniques 📊.
sammie_musyoki profile

Exploratory Data Analysis Using Data Visualization Techniques 📊.

#datascience #luxdev #beginners #dataengineering
6
Comments
5 min read
The Wrath of Unicron - When Airflow Gets Scary
elliott_cordo profile

The Wrath of Unicron - When Airflow Gets Scary

#airflow #aws #dataengineering #datamesh
6
Comments
4 min read
What Is Data Analysis and How Can You Get Started?
pavanbelagatti profile

What Is Data Analysis and How Can You Get Started?

#database #datascience #dataengineering #analytics
6
Comments
4 min read
Data Modeling
k_ndrick profile

Data Modeling

#datawarehouse #data #dataengineering #dbms
6
Comments
5 min read
My Experience with Apache Airflow
satoshi-sh profile

My Experience with Apache Airflow

#intern #airflow #python #dataengineering
6
Comments
3 min read
Engenharia de Dados com Scala: aprenda a fazer webscraping dos filmes mais assistidos da Netflix em cada país
geazi_anc profile

Engenharia de Dados com Scala: aprenda a fazer webscraping dos filmes mais assistidos da Netflix em cada país

#scala #dataengineering #braziliandevs #datascience
6
Comments 2
22 min read
Optimizing Data Analysis: A Guide to Handling Missing Data Effectively
jagroop2001 profile

Optimizing Data Analysis: A Guide to Handling Missing Data Effectively

#machinelearning #python #dataengineering #skikitlearn
6
Comments
3 min read
How to Import Existing Resources in your CloudFormation Stacks
allanchua101 profile

How to Import Existing Resources in your CloudFormation Stacks

#devops #aws #cloudformation #dataengineering
6
Comments
4 min read
Mais dados é melhor que um algoritmo mais eficiente
nobrelucas profile

Mais dados é melhor que um algoritmo mais eficiente

#datascience #data #dataengineering #braziliandevs
6
Comments
3 min read
A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes
mage_ai profile

A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes

#startup #dataengineering #datapipelines #etl
6
Comments
9 min read
Transform your Pandas Dataframes: Styles, 🎨 Colors, and 😎 Emojis
r0mymendez profile

Transform your Pandas Dataframes: Styles, 🎨 Colors, and 😎 Emojis

#datascience #python #dataengineering #data
6
Comments
9 min read
How to pivot data using Dynamic SQL in SQL Server
notte profile

How to pivot data using Dynamic SQL in SQL Server

#sql #database #dataengineering #mssql
5
Comments 4
3 min read
A Step-by-Step Guide to Implementing Data Version Control
supratipb profile

A Step-by-Step Guide to Implementing Data Version Control

#data #aws #dataengineering #database
5
Comments
4 min read
How to Convert Dates in One Group into An Interval
esproc_spl profile
Judy

How to Convert Dates in One Group into An Interval

#sql #database #dataengineering #development
5
Comments
2 min read
What is data engineering and a B.I architecture
pedrohgoncalves profile

What is data engineering and a B.I architecture

#dataengineering #datascience #beginners #architecture
5
Comments
6 min read
Data Engineering For Beginners: A Step-By-Step Guide
k_ndrick profile

Data Engineering For Beginners: A Step-By-Step Guide

#database #sql #dataengineering #datapipeline
5
Comments
8 min read
Navigating the Data Engineering Landscape: From Raw Data to Insights
rayanral profile

Navigating the Data Engineering Landscape: From Raw Data to Insights

#dataengineering #programming #learning #career
5
Comments 1
7 min read
What's new and noteworthy on AWS - Summer 2023 edition
pmalmirae profile

What's new and noteworthy on AWS - Summer 2023 edition

#dataengineering #serverless #devops #security
5
Comments
24 min read
Building ETL/ELT Pipelines For Data Engineers.
wanjohichristopher profile

Building ETL/ELT Pipelines For Data Engineers.

#snowflake #dbt #airflow #dataengineering
5
Comments 2
2 min read
Xavier's Insight: Overcoming Data Hoarding Disorder
balagmadhu profile

Xavier's Insight: Overcoming Data Hoarding Disorder

#discuss #dataengineering
5
Comments
3 min read
Automating Talend Jobs Using Apache Airflow .
wanjohichristopher profile

Automating Talend Jobs Using Apache Airflow .

#airflow #dataengineering #automation #data
5
Comments
3 min read
Handling NULL in the DBs
irinabezgina profile

Handling NULL in the DBs

#database #dataengineering #sql
5
Comments 1
2 min read
Data teams can deliver 10x better to the rest of us
mzsrtgzr2 profile

Data teams can deliver 10x better to the rest of us

#dataengineering #productivity #terminal #softwareengineering
5
Comments
3 min read
How to build an Anomaly Detector using BigQuery
chloecaroneng profile

How to build an Anomaly Detector using BigQuery

#dataengineering #datascience #bigquery #machinelearning
4
Comments
12 min read
Data Evolution - Databases to Data Lakehouse
adnankhanxx profile

Data Evolution - Databases to Data Lakehouse

#aws #dataengineering #database #datalake
4
Comments
4 min read
The Importance of Data in Decision Making
shaheryaryousaf profile

The Importance of Data in Decision Making

#data #datascience #dataengineering #ai
4
Comments
2 min read
Amazon Kinesis Data Streams (What is?, Benefits, Terminologies)
ayad profile

Amazon Kinesis Data Streams (What is?, Benefits, Terminologies)

#dataengineering #aws #cloud #serverless
4
Comments
2 min read
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
rafaelpierre profile

PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

#spark #dataengineering #airflow #orchestration
4
Comments
1 min read
SQL + Docker: The combo for Quick and Safe Query Testing
bricefotzo profile

SQL + Docker: The combo for Quick and Safe Query Testing

#devops #sql #dataengineering #docker
4
Comments
5 min read
Data Version Control vs. Open Table Formats: Differences And Use Cases
iddoavn profile

Data Version Control vs. Open Table Formats: Differences And Use Cases

#database #git #dataengineering
4
Comments
5 min read
Benchmarking Python Processing Engines: Who’s the Fastest?
bruno-nirello profile

Benchmarking Python Processing Engines: Who’s the Fastest?

#duckdb #python #dataengineering #database
3
Comments
4 min read
Transactions and the ACID principle, going a little deeper.
pedrohgoncalves profile

Transactions and the ACID principle, going a little deeper.

#database #beginners #acid #dataengineering
3
Comments
11 min read
KNIME Analytics Platform for Data Science-1
chaminduperera profile

KNIME Analytics Platform for Data Science-1

#datascience #dataengineering #ai #machinelearning
3
Comments
4 min read
🦿🛴Smarcity garbage reporting automation w/ ollama
adriens profile

🦿🛴Smarcity garbage reporting automation w/ ollama

#ai #opensource #automation #dataengineering
3
Comments 4
3 min read
The Mythical Data Team
burakkarakan profile

The Mythical Data Team

#data #dataengineering #team #datascience
3
Comments
6 min read
loading...