DEV Community

loading...

# spark

👋 Sign in for the ability sort posts by top and latest.
How-to guide: Set up, Manage & Monitor Spark on Kubernetes

How-to guide: Set up, Manage & Monitor Spark on Kubernetes

Reactions 17
10 min read
Spark and Docker: Your Spark development cycle just got 10x faster !

Spark and Docker: Your Spark development cycle just got 10x faster !

Reactions 13
7 min read
Apache Spark Java Tutorial: Simplest Guide to Get Started

Apache Spark Java Tutorial: Simplest Guide to Get Started

Reactions 6
3 min read
Is Structured Streaming Exactly-Once? Well, it depends...

Is Structured Streaming Exactly-Once? Well, it depends...

Reactions 6
4 min read
can a map function be executed on multiple executors for an item in RDD.

can a map function be executed on multiple executors for an item in RDD.

Reactions 3
1 min read
Predicting machine failures with distributed computing (Spark, AWS EMR, and DL)

Predicting machine failures with distributed computing (Spark, AWS EMR, and DL)

Reactions 8
10 min read
Using Aerospike Connect For Spark

Using Aerospike Connect For Spark

Reactions 6
5 min read
Migrating from a plain Spark Application to ZIO with ZparkIO

Migrating from a plain Spark Application to ZIO with ZparkIO

Reactions 9
6 min read
Spark: unit, integration and end-to-end tests.

Spark: unit, integration and end-to-end tests.

Reactions 12
5 min read
Spark Side Menu Micro-Interactions Deconstruction

Spark Side Menu Micro-Interactions Deconstruction

Reactions 2
2 min read
Spark Journey begins...

Spark Journey begins...

Reactions 6
3 min read
Working with nested structures in Spark

Working with nested structures in Spark

Reactions 6 Comments 1
3 min read
Intoduction to Apache Spark

Intoduction to Apache Spark

Reactions 8
6 min read
Large-Scale Data Quality Verification in .NET PT.1

Large-Scale Data Quality Verification in .NET PT.1

Reactions 2
9 min read
Unit Testing Apache Spark Structured Streaming using MemoryStream

Unit Testing Apache Spark Structured Streaming using MemoryStream

Reactions 7
4 min read
Exploiting Schema Inference in Apache Spark

Exploiting Schema Inference in Apache Spark

Reactions 2
3 min read
Setting up IntelliJ IDEA for Apache Spark and Scala development

Setting up IntelliJ IDEA for Apache Spark and Scala development

Reactions 3
2 min read
How to make a column non-nullable in Spark Structured Streaming

How to make a column non-nullable in Spark Structured Streaming

Reactions 3
2 min read
Hadoop vs Spark: Which is a better framework to select for processing Big Data?

Hadoop vs Spark: Which is a better framework to select for processing Big Data?

Reactions 4
5 min read
Why are we building DevOps platform for Big Data?

Why are we building DevOps platform for Big Data?

Reactions 3
3 min read
The Big Data Bravura: Introducing Apache Spark

The Big Data Bravura: Introducing Apache Spark

Reactions 20 Comments 2
3 min read
Install Apache Spark (and Apache Hadoop) smoothly

Install Apache Spark (and Apache Hadoop) smoothly

Reactions 8
1 min read
Apache Spark and Databricks 101 pt. II - Some DataFrames

Apache Spark and Databricks 101 pt. II - Some DataFrames

Reactions 2
1 min read
When To Cache?

When To Cache?

Reactions 5
2 min read
Apache Spark and Databricks 101 pt. I - The Big Picture

Apache Spark and Databricks 101 pt. I - The Big Picture

Reactions 3
2 min read
On.NET Episode: Data processing with .NET for Apache Spark

On.NET Episode: Data processing with .NET for Apache Spark

Reactions 7
1 min read
Python, Spark and the JVM: An overview of the PySpark Runtime Architecture

Python, Spark and the JVM: An overview of the PySpark Runtime Architecture

Reactions 10
4 min read
How to compare your data in/with Spark

How to compare your data in/with Spark

Reactions 6
6 min read
Writing Spark: Scala Vs Java

Writing Spark: Scala Vs Java

Reactions 9 Comments 2
7 min read
The 5-minute guide to using bucketing in Pyspark

The 5-minute guide to using bucketing in Pyspark

Reactions 8 Comments 4
4 min read
spark-submit command builder with live preview

spark-submit command builder with live preview

Reactions 8
1 min read
Databricks Delta Lake - A Friendly Intro

Databricks Delta Lake - A Friendly Intro

Reactions 10
1 min read
How to view Spark History logs locally

How to view Spark History logs locally

Reactions 4
1 min read
How to run pyspark with additional Spark packages

How to run pyspark with additional Spark packages

Reactions 6
2 min read
Installing and Running Hadoop and Spark on Ubuntu 18

Installing and Running Hadoop and Spark on Ubuntu 18

Reactions 27 Comments 5
10 min read
Types of Apache Spark tables and views

Types of Apache Spark tables and views

Reactions 9
2 min read
Yet another journey to Cloudera Spark and Hadoop Developer Certification - CCA 175

Yet another journey to Cloudera Spark and Hadoop Developer Certification - CCA 175

Reactions 8
6 min read
Path to become a junior+ data engineer?

Path to become a junior+ data engineer?

Reactions 5 Comments 1
1 min read
Introduction to Apache Spark

Introduction to Apache Spark

Reactions 8
3 min read
Azure Blob Storage with Pyspark

Azure Blob Storage with Pyspark

Reactions 10 Comments 1
2 min read
Why we chose Apache Spark for ETL (Extract-Transform-Load)

Why we chose Apache Spark for ETL (Extract-Transform-Load)

Reactions 23
6 min read
Divide RDD into sub parts

Divide RDD into sub parts

Reactions 5
2 min read
Big Data file formats explained

Big Data file formats explained

Reactions 10
7 min read
Spark. Anatomy of Spark application

Spark. Anatomy of Spark application

Reactions 13
6 min read
My first experience with SPARK-Ada

My first experience with SPARK-Ada

Reactions 8 Comments 4
6 min read
[Antisèche] Apache Spark : structure d'une application Spark

[Antisèche] Apache Spark : structure d'une application Spark

Reactions 6
2 min read
Live notetaking as I learn Spark

Live notetaking as I learn Spark

Reactions 24 Comments 2
11 min read
Different ways to word count in apache spark

Different ways to word count in apache spark

Reactions 9
2 min read
Big Data Analysis with Hadoop, Spark, and R Shiny

Big Data Analysis with Hadoop, Spark, and R Shiny

Reactions 28 Comments 1
12 min read
Installing and Running Hadoop and Spark on Windows

Installing and Running Hadoop and Spark on Windows

Reactions 46 Comments 58
8 min read
Processing Streaming Twitter Data using Kafka and Spark - Part 2: Creating Kafka Twitter producer

Processing Streaming Twitter Data using Kafka and Spark - Part 2: Creating Kafka Twitter producer

Reactions 21 Comments 5
7 min read
Processing Streaming Twitter Data using Kafka and Spark — The Plan

Processing Streaming Twitter Data using Kafka and Spark — The Plan

Reactions 10
2 min read
Apache Livy - Apache Spark, HDFS, and Kerberos

Apache Livy - Apache Spark, HDFS, and Kerberos

Reactions 13
2 min read
Monitoring Data Quality in Data Science Applications

Monitoring Data Quality in Data Science Applications

Reactions 27
8 min read
Learning Scala for Spark, or, what's up with that triple equals?

Learning Scala for Spark, or, what's up with that triple equals?

Reactions 16
2 min read
Apache Spark vs. Apache Flink

Apache Spark vs. Apache Flink

Reactions 31 Comments 3
6 min read
Graph Theory and Network Science for Natural Language Processing – Part 2, Databases and Analytics Engines

Graph Theory and Network Science for Natural Language Processing – Part 2, Databases and Analytics Engines

Reactions 2
6 min read
How to create a low-cost Apache Spark cluster on Microsoft Azure

How to create a low-cost Apache Spark cluster on Microsoft Azure

Reactions 6
4 min read
Configuring an Azure VNET to use AZTK in mixed mode

Configuring an Azure VNET to use AZTK in mixed mode

Reactions 6
3 min read
Proving the correctness of a binary search procedure with SPARK/Ada

Proving the correctness of a binary search procedure with SPARK/Ada

Reactions 6
9 min read
loading...