DEV Community

# dataengineering

Posts

ūüĎč Sign in for the ability to sort posts by relevant, latest, or top.
What is the Lakehouse, the latest Direction of Big Data Architecture?

What is the Lakehouse, the latest Direction of Big Data Architecture?

Reactions 8 Comments
10 min read
Correlation With Rapid Miner

Correlation With Rapid Miner

Reactions 6 Comments
1 min read
SQL FUNCTION WHERE(How to get the rows you want)

SQL FUNCTION WHERE(How to get the rows you want)

Reactions 7 Comments
1 min read
Data Preparation using Rapid miner

Data Preparation using Rapid miner

Reactions 4 Comments
1 min read
SQL Practice Exercises

SQL Practice Exercises

Reactions 10 Comments
1 min read
How to prepare for the GCP Professional Data Engineer certification

How to prepare for the GCP Professional Data Engineer certification

Reactions 16 Comments
8 min read
Making Data Engineering Easier: Operational Analytics With Event Streaming and Reverse ETL

Making Data Engineering Easier: Operational Analytics With Event Streaming and Reverse ETL

Reactions 7 Comments
6 min read
Using dbt for Transformation Tasks on BigQuery

Using dbt for Transformation Tasks on BigQuery

Reactions 10 Comments 1
4 min read
Considerations when performing ETL

Considerations when performing ETL

Reactions 4 Comments
3 min read
Docker and Kubernetes

Docker and Kubernetes

Reactions 5 Comments
3 min read
How to Use Apache Airflow to Get 1000+ Files From a Public Dataset

How to Use Apache Airflow to Get 1000+ Files From a Public Dataset

Reactions 7 Comments
10 min read
ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)

ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)

Reactions 7 Comments
5 min read
What is Azure Synapse Analytics?

What is Azure Synapse Analytics?

Reactions 4 Comments
7 min read
Design concept of a best opensource project about big data and data lakehouse

Design concept of a best opensource project about big data and data lakehouse

Reactions 9 Comments
9 min read
Details of 4 best opensource projects about big data you should try outÔľą‚Ö†ÔľČ

Details of 4 best opensource projects about big data you should try outÔľą‚Ö†ÔľČ

Reactions 7 Comments
5 min read
Debezium Change Data Capture without Kafka Connect

Debezium Change Data Capture without Kafka Connect

Reactions 7 Comments
8 min read
Building GCS Buckets and BigQuery Tables with Terraform

Building GCS Buckets and BigQuery Tables with Terraform

Reactions 6 Comments
4 min read
Released yuniql v1.2.25. Multi-tenant support, Oracle and largest set of bug fixes

Released yuniql v1.2.25. Multi-tenant support, Oracle and largest set of bug fixes

Reactions 5 Comments
4 min read
Quick use of CDC: A new demo from lakesoul makes it easier to set up the environment

Quick use of CDC: A new demo from lakesoul makes it easier to set up the environment

Reactions 8 Comments
5 min read
4 best opensource projects about big data you should try out

4 best opensource projects about big data you should try out

Reactions 15 Comments 3
3 min read
Preparing for Professional Cloud Data Engineer Certification (March 2022)

Preparing for Professional Cloud Data Engineer Certification (March 2022)

Reactions 2 Comments 2
12 min read
A new unified streaming and batch table storage solution similar to iceberg/hudi/delta lake but with several new functions

A new unified streaming and batch table storage solution similar to iceberg/hudi/delta lake but with several new functions

Reactions 8 Comments
2 min read
Introducting Myself

Introducting Myself

Reactions 7 Comments 2
1 min read
[OPINIÃO] Construindo uma Carreira como Data Engineer

[OPINIÃO] Construindo uma Carreira como Data Engineer

Reactions 2 Comments
2 min read
What is Data Profiling?

What is Data Profiling?

Reactions 2 Comments
1 min read
Data architecture models

Data architecture models

Reactions 2 Comments
6 min read
[ARTIGO] Data Warehouse, Data Lake e Data Lakehouse: Conceitos e Diferenças

[ARTIGO] Data Warehouse, Data Lake e Data Lakehouse: Conceitos e Diferenças

Reactions 4 Comments
3 min read
Enabling the Customer Data Stack: RudderStack Series B Funding

Enabling the Customer Data Stack: RudderStack Series B Funding

Reactions 2 Comments
1 min read
Kestra, infinitely scalable open source orchestration and scheduling platform.

Kestra, infinitely scalable open source orchestration and scheduling platform.

Reactions 3 Comments
6 min read
Modern data warehouse patterns: ELT with Snowflake variants

Modern data warehouse patterns: ELT with Snowflake variants

Reactions 9 Comments
6 min read
Data Engineering in Julia

Data Engineering in Julia

Reactions 3 Comments
1 min read
Standing on the shoulders of giants. Part one: Airflow

Standing on the shoulders of giants. Part one: Airflow

Reactions 7 Comments
5 min read
How Engineering Teams Use RudderStack to Support Marketing

How Engineering Teams Use RudderStack to Support Marketing

Reactions 6 Comments
7 min read
Data Engineering Pipeline with AWS Step Functions, CodeBuild and Dagster

Data Engineering Pipeline with AWS Step Functions, CodeBuild and Dagster

Reactions 6 Comments 3
10 min read
Why It’s Hard for Engineering to Support Marketing

Why It’s Hard for Engineering to Support Marketing

Reactions 2 Comments
3 min read
Extract csv data and load it to PostgreSQL using Meltano ELT

Extract csv data and load it to PostgreSQL using Meltano ELT

Reactions 5 Comments
6 min read
What Is Event-Driven Machine Learning?

What Is Event-Driven Machine Learning?

Reactions 6 Comments
4 min read
Host a fully persisted Apache NiFi service with docker

Host a fully persisted Apache NiFi service with docker

Reactions 3 Comments
1 min read
Relational data models

Relational data models

Reactions 5 Comments
2 min read
Implementing Graceful Shutdown in Go

Implementing Graceful Shutdown in Go

Reactions 14 Comments 5
14 min read
Open Source Analytics Stack: Bringing Control, Flexibility, and Data-Privacy to Your Analytics

Open Source Analytics Stack: Bringing Control, Flexibility, and Data-Privacy to Your Analytics

Reactions 3 Comments
10 min read
RudderStack + Blendo: Better Together

RudderStack + Blendo: Better Together

Reactions 2 Comments
7 min read
Web Scraping Sprott U Fund with BS4 in 10 Lines of Code

Web Scraping Sprott U Fund with BS4 in 10 Lines of Code

Reactions 29 Comments
3 min read
RudderStack’s Licensing Explained

RudderStack’s Licensing Explained

Reactions 3 Comments
4 min read
Introducing RudderStack's New, High-performance JavaScript SDK

Introducing RudderStack's New, High-performance JavaScript SDK

Reactions 2 Comments
3 min read
The Open Source Story - Open Sourcing RudderStack Blog and Docs

The Open Source Story - Open Sourcing RudderStack Blog and Docs

Reactions 3 Comments
5 min read
The Data Engineering Megatrend: A Brief History

The Data Engineering Megatrend: A Brief History

Reactions 2 Comments
7 min read
RudderStack Product News Vol. #013 - Destinations Re-design and New Integrations

RudderStack Product News Vol. #013 - Destinations Re-design and New Integrations

Reactions 2 Comments
2 min read
DataEngBytes conference wrap up

DataEngBytes conference wrap up

Reactions 11 Comments
2 min read
Stream Your Database Changes with Change Data Capture: Part Two

Stream Your Database Changes with Change Data Capture: Part Two

Reactions 5 Comments
10 min read
Why the Cloud SaaS Tools Used by Marketing, Sales, and Product Teams Create Data Silos

Why the Cloud SaaS Tools Used by Marketing, Sales, and Product Teams Create Data Silos

Reactions 3 Comments
5 min read
The Data Trinity

The Data Trinity

Reactions 4 Comments
4 min read
Want To Learn MLOps?

Want To Learn MLOps?

Reactions 10 Comments
4 min read
Stream Your Database Changes with Change Data Capture

Stream Your Database Changes with Change Data Capture

Reactions 9 Comments
9 min read
Editing Tabular Data in Angular

Editing Tabular Data in Angular

Reactions 5 Comments
11 min read
Evolution of a data system

Evolution of a data system

Reactions 10 Comments 2
5 min read
Creating a Soft Delete Archive Table with PostgreSQL

Creating a Soft Delete Archive Table with PostgreSQL

Reactions 4 Comments
2 min read
I Started Learning Scala as a Python Programmer. Here’s Why.

I Started Learning Scala as a Python Programmer. Here’s Why.

Reactions 4 Comments 1
5 min read
Quick profiling of data in Apache Kafka using kafkacat and visidata

Quick profiling of data in Apache Kafka using kafkacat and visidata

Reactions 2 Comments 1
2 min read
ūüďľ ksqlDB HOWTO - A mini video series ūüďľ

ūüďľ ksqlDB HOWTO - A mini video series ūüďľ

Reactions 9 Comments
4 min read
loading...