DEV Community

# bigdata

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Data Engineering Terminology: Understanding Upstream and Downstream in Data Pipelines
luminousmen profile

Data Engineering Terminology: Understanding Upstream and Downstream in Data Pipelines

#bigdata #data
Comments
1 min read
Business Intelligence Data Analyst vs. BI Developer
ai_jobsnet profile

Business Intelligence Data Analyst vs. BI Developer

#bigdata #analyst #career #programming
2
Comments
3 min read
🏆How to master 📊 Big Data pipelines with Taipy and PySpark 🐍
marisogo profile
Marine for Taipy

🏆How to master 📊 Big Data pipelines with Taipy and PySpark 🐍

#python #opensource #bigdata #tutorial
197
Comments 8
9 min read
BigData Journey from Hadoop and MapReduce to AWS EMR
olgitz profile

BigData Journey from Hadoop and MapReduce to AWS EMR

#bigdata #emr #spark #hadoop
Comments
9 min read
From Hadoop to Cloud: Why and How to Decouple Storage and Compute in Big Data Platforms
daswu profile
DASWU

From Hadoop to Cloud: Why and How to Decouple Storage and Compute in Big Data Platforms

#opensource #bigdata
Comments
13 min read
S3 Multi-Part Upload: Part 2 Conclusion
mitanshgor profile

S3 Multi-Part Upload: Part 2 Conclusion

#aws #bigdata #s3 #multipart
5
Comments
11 min read
Big data models 📊 vs. Computer memory 💾
marisogo profile
Marine for Taipy

Big data models 📊 vs. Computer memory 💾

#bigdata #pipeline #dataengineering #dask
186
Comments 3
11 min read
Working with Parquet files in Java using Avro
jerolba profile

Working with Parquet files in Java using Avro

#parquet #java #avro #bigdata
1
Comments
10 min read
Which Scenarios Does ClickHouse Applies to?
jbx1279 profile

Which Scenarios Does ClickHouse Applies to?

#bigdata #performance #database #sql
5
Comments
9 min read
Most common errors when setting up Amazon EMR
nowsathk profile

Most common errors when setting up Amazon EMR

#emr #dynamodb #hive #bigdata
8
Comments
2 min read
HyperLogLog | Un algoritmo para contarlos (aproximadamente) a todos
javier-aranda profile

HyperLogLog | Un algoritmo para contarlos (aproximadamente) a todos

#algorithms #computerscience #bigdata #spanish
2
Comments
6 min read
Data-Powered Accessibility: How to Build Inclusive Product for Any User Need
zotkina profile

Data-Powered Accessibility: How to Build Inclusive Product for Any User Need

#datapowered #bigdata #inclusiveproduct #a11y
48
Comments
7 min read
Install Hadoop on Ubuntu
jerry-45 profile

Install Hadoop on Ubuntu

#bigdata #hadoop #ubuntu #learning
1
Comments
6 min read
Connecting Multiple Kafka Clusters in ClickHouse Using Named Collections
ranjbaryshahab profile

Connecting Multiple Kafka Clusters in ClickHouse Using Named Collections

#clickhouse #kafka #dataintegration #bigdata
4
Comments
3 min read
SPL computing performance test series: in-group accumulation
jbx1279 profile

SPL computing performance test series: in-group accumulation

#performance #bigdata #database
1
Comments
12 min read
Log Analysis: Elasticsearch VS Apache Doris
apachedoris profile

Log Analysis: Elasticsearch VS Apache Doris

#beginners #database #dataengineering #bigdata
Comments
11 min read
SPL computing performance test series: funnel analysis
jbx1279 profile

SPL computing performance test series: funnel analysis

#performance #bigdata #database
1
Comments
16 min read
SPL computing performance test series: associate tables and wide table
jbx1279 profile

SPL computing performance test series: associate tables and wide table

#database #performance #bigdata #sql
Comments
6 min read
SPL computing performance test series: position association
jbx1279 profile

SPL computing performance test series: position association

#performance #bigdata #database
1
Comments
12 min read
SPL computing performance test series: multi-index aggregating
jbx1279 profile

SPL computing performance test series: multi-index aggregating

#bigdata #database #performance #sql
1
Comments
6 min read
What is '_spark_metadata' Directory in Spark Structured Streaming ?
sukumaar profile

What is '_spark_metadata' Directory in Spark Structured Streaming ?

#apachespark #scala #java #bigdata
Comments
3 min read
SQL is consuming the lives of data scientists
jbx1279 profile

SQL is consuming the lives of data scientists

#sql #datascience #database #bigdata
6
Comments 3
20 min read
⛏ Get Mining into Data with These Top 5 Resources
evergrowingdev profile

⛏ Get Mining into Data with These Top 5 Resources

#bigdata #datascience #learning #beginners
5
Comments 2
6 min read
Apache Doris 2.0 Beta Now Available: Faster, Stabler, and More Versatile
apachedoris profile

Apache Doris 2.0 Beta Now Available: Faster, Stabler, and More Versatile

#database #datascience #opensource #bigdata
Comments
15 min read
Is Your Latest Data Really the Latest? Check the Data Update Mechanism of Your Database
apachedoris profile

Is Your Latest Data Really the Latest? Check the Data Update Mechanism of Your Database

#datascience #database #sql #bigdata
4
Comments 1
6 min read
Introduction to Big-data
razishaikh28 profile

Introduction to Big-data

#bigdata #shortoverview #shortnotes
2
Comments 2
3 min read
The performance problems of data warehouse and solutions
jbx1279 profile

The performance problems of data warehouse and solutions

#database #sql #performance #bigdata
Comments
14 min read
Snowflake: Revolutionizing data warehousing
mage_ai profile
Mage

Snowflake: Revolutionizing data warehousing

#datawarehouse #snowflake #cloudcomputing #bigdata
3
Comments
6 min read
5 Common Mistakes with Apache Flink and How to Avoid Them
umairk profile

5 Common Mistakes with Apache Flink and How to Avoid Them

#bigdata #datascience #development #beginners
2
Comments
3 min read
Next Big Data System
zhenxuanpan profile

Next Big Data System

#bigdata #java #graph #stream
Comments
1 min read
Open-source SPL： The Breaker of Closed Database Computing System
jbx1279 profile

Open-source SPL： The Breaker of Closed Database Computing System

#database #sql #bigdata #programming
Comments 1
8 min read
3 Data Observability Tools
yayabobi profile

3 Data Observability Tools

#bigdata #data #observability #devtools
Comments
3 min read
Why Are There So Many Snapshot Tables in BI Systems?
jbx1279 profile

Why Are There So Many Snapshot Tables in BI Systems?

#database #bigdata #sql #programming
5
Comments
9 min read
Why does wide table prevail?
jbx1279 profile

Why does wide table prevail?

#database #bigdata #sql #programming
5
Comments
13 min read
Bulk load to Elastic Search with PySpark
valerybriz profile

Bulk load to Elastic Search with PySpark

#elasticsearch #spark #pyspark #bigdata
2
Comments
2 min read
Routable computing engine implements front-end database
jbx1279 profile

Routable computing engine implements front-end database

#database #sql #bigdata #programming
Comments
5 min read
Processing EventHub Captured Messages in Avro Files Using Databricks
damiansmazurek profile

Processing EventHub Captured Messages in Avro Files Using Databricks

#data #azure #bigdata #programming
Comments
2 min read
How does the in-memory database bring memory’s advantage into play?
jbx1279 profile

How does the in-memory database bring memory’s advantage into play?

#database #sql #bigdata
Comments
12 min read
How to clone tables in BigQuery
mesmacosta profile

How to clone tables in BigQuery

#bigquery #bigdata #dataengineering
2
Comments
1 min read
Why ETL Becomes ELT or Even LET?
jbx1279 profile

Why ETL Becomes ELT or Even LET?

#database #bigdata #sql #programming
Comments
8 min read
HTAP: Learning from Xiaohongshu
lazypro profile

HTAP: Learning from Xiaohongshu

#architecture #bigdata #database #datascience
1
Comments
5 min read
HTAP database cannot handle HTAP requirements
jbx1279 profile

HTAP database cannot handle HTAP requirements

#database #bigdata #sql #programming
Comments
13 min read
Integrating Apache Age with Other Big Data Tools and Frameworks
mohanadtoaima profile

Integrating Apache Age with Other Big Data Tools and Frameworks

#apacheage #apache #postgres #bigdata
2
Comments 1
2 min read
The current Lakehouse is like a false proposition
jbx1279 profile

The current Lakehouse is like a false proposition

#bigdata #lakehouse #database #programming
Comments
11 min read
How to make the columnar storage data warehouse more efficient
jbx1279 profile

How to make the columnar storage data warehouse more efficient

#bigdata #database #programming
Comments
11 min read
Exploration of Spark Executor Memory
lorenzolou profile

Exploration of Spark Executor Memory

#spark #programming #bigdata
Comments
9 min read
Simplest pyspark tutorial
ndurumo254 profile

Simplest pyspark tutorial

#spark #bigdata #machinelearning #sql
2
Comments
7 min read
Making Debezium 2.x Support Confluent Schema Registry
lazypro profile

Making Debezium 2.x Support Confluent Schema Registry

#bigdata #architecture #docker #tutorial
1
Comments 3
3 min read
Performance Enhancement: Conversion Funnel Analysis
jbx1279 profile

Performance Enhancement: Conversion Funnel Analysis

#bigdata #sql #database #programming
Comments
9 min read
Boost Your Testing Strategy: The Coolest Methods to Prioritize A/B Tests Like a Pro! 🎲📊😎
olga_r profile

Boost Your Testing Strategy: The Coolest Methods to Prioritize A/B Tests Like a Pro! 🎲📊😎

#analytics #bigdata #productivity
3
Comments
4 min read
A Comprehensive Comparison of JuiceFS and HDFS for Cloud-Based Big Data Storage
tonybarber2 profile

A Comprehensive Comparison of JuiceFS and HDFS for Cloud-Based Big Data Storage

#bigdata #opensource #cloud
1
Comments
11 min read
How to use docker to compile Apache Doris
lemonlitree profile

How to use docker to compile Apache Doris

#apachedoris #doris #olap #bigdata
2
Comments
3 min read
Apache Doris be common problem positioning and processing
lemonlitree profile

Apache Doris be common problem positioning and processing

#apachedoris #doris #olap #bigdata
1
Comments
3 min read
The Secret to Rapid Scaling: How Scraping Helped These Startups Go From Zero to $1.2+ Trillion
tomaslau profile

The Secret to Rapid Scaling: How Scraping Helped These Startups Go From Zero to $1.2+ Trillion

#startup #bigdata #scraping #datascience
6
Comments 1
6 min read
Mastering Large-Scale Data Processing: Building a Data Pipeline with ApacheAGE for Efficient Ingestion, Processing, and Analysis
humzakt profile

Mastering Large-Scale Data Processing: Building a Data Pipeline with ApacheAGE for Efficient Ingestion, Processing, and Analysis

#apacheage #postgres #datascience #bigdata
2
Comments
2 min read
How we mastered dbt: A true story
olgabraginskaya profile

How we mastered dbt: A true story

#bigdata #dataengineering #dbt #tutorial
7
Comments
14 min read
GETTING STARTED WITH SENTIMENT ANALYSIS.
brieatieno profile

GETTING STARTED WITH SENTIMENT ANALYSIS.

#python #datascience #dataanalysis #bigdata
2
Comments
4 min read
Lightweight HTTP API for Big Data on S3
pdambrauskas profile

Lightweight HTTP API for Big Data on S3

#deltalake #bigdata #opensource #s3
3
Comments
3 min read
How to cope with high-concurrency account query?
jbx1279 profile

How to cope with high-concurrency account query?

#database #bigdata #performance #programming
Comments
6 min read
Don't Break the Bank on SQL Queries: BigQuery On-Demand vs Flat-Rate prices. Which Saves You More? 💰😎
olga_r profile

Don't Break the Bank on SQL Queries: BigQuery On-Demand vs Flat-Rate prices. Which Saves You More? 💰😎

#productivity #database #sql #bigdata
5
Comments 3
5 min read
loading...