Dataengineering

1 min read

BenBirt for Dataform

Sep 24

Cut data warehouse costs with run caching

#elt #dataengineering #pipeline #etl

3 min read

Michiel Ghyselinck

Oct 15

Dagster with User Code Deployments (gRPC)

#dataengineering #etl #dagster #kubernetes

6 min read

Robin Moffatt

Sep 25

Some of my favourite public data sets

#opendata #data #dataengineering #datascience

8 2

2 min read

Bartosz Gajda

Sep 19

5 Essential skills for becoming a Data Engineer

#beginners #datascience #career #dataengineering

6 min read

Greg

Sep 17

The Most Popular Data Science Newsletters

#datascience #machinelearning #dataengineering

9 min read

Simon Yu

Sep 17

Build a monitored code-based pipeline to move data from Postgres to Snowflake

#aws #postgres #snowflake #dataengineering

9 min read

Wai Yan

Sep 14

Handling upstream data changes via Change Data Capture

#dataengineering #database #datapipeline

8 min read

Manikantha Nekkalapudi

Sep 14

Intoduction to Apache Spark

#dataengineering #apachespark #bigdata #spark

6 min read

01:00

Robin Moffatt

Sep 11

Kafka Connect in 60 seconds

#apachekafka #dataengineering #bigdata #dataintegration

2 min read

Simon Yu

Aug 25

Deploying data pipelines to AWS Fargate - with monitoring and alerts built-in

#aws #dataengineering #serverless #docker

3

3 min read

Samuele Resca

Sep 2

Large-Scale Data Quality Verification in .NET PT.1

#dotnet #spark #dotnetcore #dataengineering

9 min read

Josh Holbrook

Jul 12

How To Run Airflow on Windows (with Docker)

#dataengineering #etl #airflow

9 1

8 min read

Joseph

May 28

Data Engineering Project for Beginners - Batch edition

#dataengineering #tutorial #beginners #aws

8

19 min read

CitizenK

Apr 23

Airflow UI with Role-Based Access Control

#python #apacheairflow #dataengineering

3

1 min read

Waylon Walker

Mar 2

🛢Create New Kedro Pipeline (kedro new)

#data #dataengineering #kedro #datascience

4 min read

Waylon Walker

Feb 24

🤷‍♀️ What is Kedro (The Parts)

#data #dataengineering #kedro #datascience

14 3

3 min read

Josh Yap

Feb 12

Data engineering portfolio projects?

#dataengineering #data #portfolio

12 1

1 min read

Zahidul Islam

Jan 3

Apache Airflow Core Concepts

#airflow #workflow #dataengineering

15

4 min read

Oct 18 '19

5 Considerations to have when using Airflow

#analytics #machinelearning #data #dataengineering

10

6 min read

00:31

Yan Parker

Sep 14 '19

Data Engineering Skills

#dataengineering #datascience

12

1 min read

Jun 25 '19

Manage Data Pipelines with Apache Airflow

#apache #python #dataengineering #etl

62

13 min read

Luciano Strika

Apr 14 '19

How to Run Parallel Data Analysis in Python using Dask Dataframes

#dataanalysis #programming #dataengineering #parallel

6 min read

Aug 8

Scrape Structured Data with Python and Extruct

#python #scraping #dataengineering #scraper

16 min read

Robin Moffatt

Jun 23

Loading CSV data into Kafka - video walkthrough

#apachekafka #tutorial #csv #dataengineering

10 min read

BenBirt for Dataform

Jun 8

CI/CD for ETL/ELT pipelines

#cicd #etl #elt #dataengineering

18

3 min read

Data Unbound

Jun 24

Terraform in Anger Part 1: AWS S3 Access

#dataengineering #devops #aws #programming

9 min read

Kan Ouivirach for Data Eng Thailand

Jun 20

5 Challenges ในการสร้าง Production-Grade Data Pipeline

#dataengineering #datapipeline

26 5

1 min read

Krithika

Jun 21

What differentiates schema on read from schema on write?

#database #data #dataengineering #schema

2 2

3 min read

Jun 11

Scraping Data on the Web with BeautifulSoup

#python #scrapers #data #dataengineering

30

12 min read

Joseph

May 11

10 Key skills, to help you become a data engineer

#dataengineering #beginners #database #etl

3 min read

CitizenK

Apr 22

Apache Airflow Installation - mysql+celery

#python #apacheairflow #dataengineering

3

1 min read

Mar 26

Extract Nested Data From Complex JSON

#python #restapis #dataengineering

8

6 min read

Feb 13

Psycopg2: PostgreSQL & Python (the Old Fashioned Way)

#postgres #python #sql #dataengineering

16

6 min read

Nov 1 '19

Azure Message Brokers patterns for Data Applications

#datascience #dataengineering #bigdata #azure

6 min read

Luciano Strika

Oct 19 '19

Coding MapReduce in C from Scratch using Threads: Map

#programming #beginners #c #dataengineering

9 min read

Oct 19 '19

How to collect the data you need to bootstrap your digital marketing analytics

#dataengineering #data #marketing #analytics

12

12 min read

Oct 10 '19

Structured Streaming in PySpark

#spark #apache #python #dataengineering

10

9 min read

Oct 8 '19

Becoming Familiar with Apache Kafka and Message Queues

#apache #dataengineering #etl

16

6 min read

Caleb Ariel

Sep 30 '19

I am a junior data engineer without a senior engineer. What should I do?

#startup #dataengineering

6 1

1 min read

Sumit Kumar

Sep 21 '19

Why we chose Apache Spark for ETL (Extract-Transform-Load)

#spark #dataengineering #pyspark #whywechose

22

6 min read

Flo

Aug 9 '19

Intro to Data Ingestion and Data Lakes

#dataengineering #datalake #dataingestion

3 min read

Yan Parker

Aug 7 '19

Data Engineering — Complete Reference Guide From A-Z [2019]

#dataengineering #datascience #bigdata

18

16 min read

Aug 6 '19

Overview of the different approaches to putting Machine Learning (ML) models in production

#data #dataengineering #machinelearning #software

14 min read

Jul 30 '19

ON the evolution of Data Engineering

#data #dataengineering #etl #sql

14

4 min read

Will Velida

Jul 15 '19

Understanding and Optimizing Throughput in Azure Cosmos DB

#database #dataengineering #bigdata #azure

7 2

8 min read

Feb 26 '19

Welcome to SQL: Modifying Databases and Tables

#sql #dataanalysis #dataengineering #postgres

10 min read

Jan 25 '19

From CSVs to Tables: Infer Schema Data Types From Raw Spreadsheets

#bigdata #python #dataengineering #sql

8 min read

Jan 21 '19

Intro to Python Database Management with SQLAlchemy

#sqlalchemy #sql #python #dataengineering

15

7 min read

Sumit Kumar

Jan 19 '19

Know Your Data (KYD)

#dataengineering #data #datamodeling #knowyourdata

1 min read

Sumit Kumar

Jan 19 '19

Optimising E-Commerce Data - Know Your Data (KYD)

#knowyourdata #mongodb #ecommercesolution #dataengineering

3 min read

Vicki Boykis

Jun 26 '17

So, you want to data science

#datascience #dataengineering #architecture #strategy

28 2

8 min read

Sena Heydari for DoSomething.org

Jun 17 '17

Choosing Your Data Warehouse

#data #nonprofit #devops #dataengineering

10 4

1 min read

Jesse Anderson

Feb 3 '17

What Do I Look for in Data Engineers?

#dataengineering #dataengineer #bigdata

0

2 min read

Darsh Shukla

Aug 11

Data Warehouse - The Minimal Architectural Approach

#dataengineering #datascience #datawarehouse #cloud

2 min read

Lolla Paz

Jul 9

Thermal Data

#hotdata #colddata #dataengineering #data