Read CSV file:
df = spark.read.format("CSV").option("header","true").load(input_path)
df = df.na.drop(how="any")
To save as output files
data.coalesce(1).write.mode("overwrite").csv(output_path,header=True)
Spark by examples
Read CSV file:
df = spark.read.format("CSV").option("header","true").load(input_path)
df = df.na.drop(how="any")
To save as output files
data.coalesce(1).write.mode("overwrite").csv(output_path,header=True)
Spark by examples
For further actions, you may consider blocking this person and/or reporting abuse
Luciano Jung -
Nikhil Soman Sahu -
Serdar Torlak -
tosynthegeek -
Top comments (0)