Skip to content
Search

DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Batch Processing vs Stream Processing: Why Batch is dying and Streaming takes over
atrifsik profile

Batch Processing vs Stream Processing: Why Batch is dying and Streaming takes over

#batchprocessing #streamprocessing #dataengineering
Comments
13 min read
Structure Query Language
zapeeosheikh profile

Structure Query Language

#sql #data #dataengineering #blog
6
Comments
2 min read
Important Questions related to Data Engineering
zapeeosheikh profile

Important Questions related to Data Engineering

#data #dataengineering #pipelines #questions
2
Comments
1 min read
AWS Cloud9 for Data Engineers
gauravthalpati profile

AWS Cloud9 for Data Engineers

#dataengineering #aws
1
Comments
5 min read
How we mastered dbt: A true story
olgabraginskaya profile

How we mastered dbt: A true story

#bigdata #dataengineering #dbt #tutorial
3
Comments
14 min read
Using python dictionary in data engineering.
ndurumo254 profile

Using python dictionary in data engineering.

#dataengineering #python #tutorial #100daysofcode
2
Comments 2
2 min read
Integrando uma Web API com Datastore Emulator
geazi_anc profile

Integrando uma Web API com Datastore Emulator

#python #gcp #dataengineering #webdev
Comments
4 min read
Python functions and lambda functions in data engineering.
ndurumo254 profile

Python functions and lambda functions in data engineering.

#dataengineering #python #lambda #functions
2
Comments
3 min read
Data Wrangling in Python: Tips and Tricks
peterabel profile

Data Wrangling in Python: Tips and Tricks

#python #datascience #dataengineering
Comments
3 min read
How I Decreased ETL Cost by Leveraging the Apache Arrow Ecosystem
auyer profile

How I Decreased ETL Cost by Leveraging the Apache Arrow Ecosystem

#etl #python #data #dataengineering
Comments
6 min read
Data Platform Architecture Types
data_mike profile

Data Platform Architecture Types

#dataengineering #database #datascience #architecture
1
Comments
9 min read
Creating Data Pipelines as DAGs in Apache Airflow (Part 1)
franklinobasy profile

Creating Data Pipelines as DAGs in Apache Airflow (Part 1)

#dataengineering #airflow #dags #python
Comments
6 min read
Website Monitoring using AWS Lambda and Aurora
flynntran3103 profile

Website Monitoring using AWS Lambda and Aurora

#aws #datascience #dataengineering #programming
2
Comments
4 min read
Apache Airflow - Deep Dive | All you need to know about Airflow
alikhyar profile

Apache Airflow - Deep Dive | All you need to know about Airflow

#airflow #dataengineering #datascience #apacheairflow
5
Comments
20 min read
SQL101: Introduction to SQL
marrie profile

SQL101: Introduction to SQL

#sql #datascience #database #dataengineering
Comments 2
12 min read
Data Pipelines with Great Expectations | Introduction
samuelearl profile

Data Pipelines with Great Expectations | Introduction

#datascience #dataengineering #datavalidation #greatexpectations
2
Comments
2 min read
22 Best DataOps Tools To Optimize Your Data Management and Observability In 2023
pramit_marattha profile

22 Best DataOps Tools To Optimize Your Data Management and Observability In 2023

#dataops #data #beginners #dataengineering
16
Comments 1
30 min read
Building a Data Lakehouse for Analyzing Elon Musk Tweets using MinIO, Apache Airflow, Apache Drill and Apache Superset
mikekenneth77 profile

Building a Data Lakehouse for Analyzing Elon Musk Tweets using MinIO, Apache Airflow, Apache Drill and Apache Superset

#dataengineering #datascience #python #database
13
Comments 2
8 min read
Nesting Columns like a Pro: A Guide to Mastering Nested Structs in PySpark
paulopu profile

Nesting Columns like a Pro: A Guide to Mastering Nested Structs in PySpark

#python #pyspark #bigdata #dataengineering
Comments
4 min read
AWS Data Engineering Services: Everything you need to know
parthsoni profile

AWS Data Engineering Services: Everything you need to know

#aws #cloudskills #dataengineering #awscommunity
5
Comments
9 min read
PySpark: A brief analysis to the most common words in Dracula, by Bram Stoker
geazi_anc profile

PySpark: A brief analysis to the most common words in Dracula, by Bram Stoker

#python #dataengineering #spark #datascience
13
Comments
5 min read
Working with Map() function in Python, Pyspark and Apache Beam
rumsinha profile

Working with Map() function in Python, Pyspark and Apache Beam

#python #pyspark #distributedcomputing #dataengineering
1
Comments
3 min read
Working with large CSV files in Python from Scratch
ramsescoraspe profile

Working with large CSV files in Python from Scratch

#datascience #dataengineering #bigdata #python
6
Comments
1 min read
Job Search API
anuoluwapods profile

Job Search API

#api #job #dataengineering #jobsearch
6
Comments
1 min read
Redshift Deep Dive
sanjaykmenon profile

Redshift Deep Dive

#aws #redshift #dataengineering #dataanalytics
1
Comments
5 min read
Azure Data Factory - Incrementally load data from Azure SQL to Azure Data Lake using Watermark
balramprasad profile

Azure Data Factory - Incrementally load data from Azure SQL to Azure Data Lake using Watermark

#azure #bigdata #azuredatafactory #dataengineering
4
Comments
1 min read
What is data integration?
mage_ai profile

What is data integration?

#dataintegration #datapipelines #datascience #dataengineering
10
Comments 2
4 min read
Data Engineering Trends for 2023
atrifsik profile

Data Engineering Trends for 2023

#dataengineering #datacontracts #eventsourcing #streamingdata
3
Comments
4 min read
The Changing Face Of ETL
atrifsik profile

The Changing Face Of ETL

#etl #dataengineering #datapipelines #elt
3
Comments 1
12 min read
Ultimate guide to becoming a Data Analyst/Data Scientist
jamesoyanna profile

Ultimate guide to becoming a Data Analyst/Data Scientist

#datascience #dataanalyst #dataengineering #machinelearning
3
Comments
4 min read
SkyX: desenvolvimento de uma análise de tráfego aéreo em tempo real com Spark Structured Streaming e Apache Kafka
geazi_anc profile

SkyX: desenvolvimento de uma análise de tráfego aéreo em tempo real com Spark Structured Streaming e Apache Kafka

#python #sparkstructuredstreaming #kafka #dataengineering
1
Comments
8 min read
2022 Beginner Friendly Modern Data Engineering Career path With Learning Resources.
grayhat profile

2022 Beginner Friendly Modern Data Engineering Career path With Learning Resources.

#python #datascience #dataengineering #machinelearning
20
Comments 2
2 min read
Learn Ansible and how to Install it in Ubuntu 22.04.
kinyungu_denis profile

Learn Ansible and how to Install it in Ubuntu 22.04.

#tutorial #install #dataengineering
Comments
3 min read
Uma breve Introdução ao processamento de dados em tempo real com Spark Structured Streaming e Apache Kafka
geazi_anc profile

Uma breve Introdução ao processamento de dados em tempo real com Spark Structured Streaming e Apache Kafka

#python #pyspark #dataengineering #apachekafka
4
Comments
8 min read
Apache-Spark introduction for SQL developers
cesar_mostacero profile

Apache-Spark introduction for SQL developers

#apachespark #dataengineering #beginners #bigdata
2
Comments
7 min read
PySpark: uma breve análise das palavras mais comuns em Drácula, por Bram Stoker
geazi_anc profile

PySpark: uma breve análise das palavras mais comuns em Drácula, por Bram Stoker

#python #dataengineering #spark #datascience
4
Comments 6
6 min read
INTRODUCTION TO PYTHON FOR DATA ENGINEERING
fatumakaliku profile

INTRODUCTION TO PYTHON FOR DATA ENGINEERING

#python #data #dataengineering #beginners
Comments
4 min read
Create Jira Ticket on Prefect Task Failure
codiefz profile

Create Jira Ticket on Prefect Task Failure

#python #dataengineering
Comments
2 min read
Introdução à análise de dados com PySpark utilizando os dados dos campeões de League of Legends
geazi_anc profile

Introdução à análise de dados com PySpark utilizando os dados dos campeões de League of Legends

#pyspark #python #dataanalysis #dataengineering
3
Comments
8 min read
Pokemons Flow: desenvolvendo uma pipeline de dados com apache airflow para extração de pokemon via API
geazi_anc profile

Pokemons Flow: desenvolvendo uma pipeline de dados com apache airflow para extração de pokemon via API

#python #datapipeline #apacheairflow #dataengineering
9
Comments
6 min read
Apache PySpark for Data Engineering
kinyungu_denis profile

Apache PySpark for Data Engineering

#beginners #python #dataengineering #sql
6
Comments 4
9 min read
Data Engineering 101: Introduction to Data Engineering
ibrahimabdike profile

Data Engineering 101: Introduction to Data Engineering

#dataengineering #datascience #beginners #codenewbie
10
Comments
3 min read
Introduction to Python for Data Engineering
thyalpha001 profile

Introduction to Python for Data Engineering

#python #dataengineering #pythonfordataengineering #beginners
4
Comments
5 min read
Kubernetes Was Never Designed for Batch Jobs
hrichardlee profile

Kubernetes Was Never Designed for Batch Jobs

#airflow #mlops #dataengineering #kubernetes
3
Comments 2
17 min read
Data Engineering 102: Introduction to Python for Data Engineering.
kinyungu_denis profile

Data Engineering 102: Introduction to Python for Data Engineering.

#python #dataengineering #beginners
5
Comments
10 min read
Introduction to Python for Data Engineering
ndurumo254 profile

Introduction to Python for Data Engineering

#python #dataengineering #aws #iot
4
Comments
7 min read
Data Engineering 102: Introduction to Python for Data Engineering
wachuka_james profile

Data Engineering 102: Introduction to Python for Data Engineering

#python #pyspark #dataengineering
Comments
3 min read
Python For Data Engineering
brianmk profile

Python For Data Engineering

#dataengineering #dsea #python #bigdata
9
Comments
5 min read
DATA ENGINEERING 101:INTRODUCTION TO DATA ENGINNERING.
violakithinji profile

DATA ENGINEERING 101:INTRODUCTION TO DATA ENGINNERING.

#codenewbie #dataengineering #beginners #programming
5
Comments
2 min read
Online SQL Client for low code data management
pavlopaska profile

Online SQL Client for low code data management

#sql #database #dataengineering #lowcode
5
Comments 1
5 min read
Data Engineering 101: Introduction to Data Engineering.
kinyungu_denis profile

Data Engineering 101: Introduction to Data Engineering.

#dataengineering #beginners
3
Comments
6 min read
Hash Personal Identifiable Information (PII) in your ELT pipelines
codiefz profile

Hash Personal Identifiable Information (PII) in your ELT pipelines

#python #dataengineering #database
3
Comments
3 min read
Difference Between Data Engineer and Data Scientist?
zapeeosheikh profile

Difference Between Data Engineer and Data Scientist?

#difference #pipeline #dataengineering #datascientist
7
Comments
3 min read
Learning Workflow Schedulers (Oozie)
liruikaigx profile

Learning Workflow Schedulers (Oozie)

#bigdata #datascience #dataengineering
1
Comments
5 min read
Solving AttributeError: 'float' object has no attribute 'rint'
theolakusibe profile

Solving AttributeError: 'float' object has no attribute 'rint'

#python #etl #dataengineering #pandas
3
Comments
2 min read
[Spark-k8s] — Getting started # Part 1
tiagotxm profile

[Spark-k8s] — Getting started # Part 1

#spark #kubernetes #dataengineering
1
Comments
4 min read
Websites to find Dataset for your Data Engineering projects.
saifullah profile

Websites to find Dataset for your Data Engineering projects.

#datascience #dataengineering #database #dataset
5
Comments
1 min read
Data engineers must-see: The future trend of big data cloud services
dmetasoul profile

Data engineers must-see: The future trend of big data cloud services

#database #dataengineering #bigdata #opensource
8
Comments
8 min read
Data Engineering Projects for Beginners
ramsescoraspe profile

Data Engineering Projects for Beginners

#database #dataengineering #python
17
Comments 2
2 min read
Data Pipelines with Apache Airflow - Book Review
albertulysses profile

Data Pipelines with Apache Airflow - Book Review

#python #dataengineering #books #bigdata
6
Comments
2 min read
loading...