DEV Community 👩‍💻👨‍💻

Cover image for DP-900 Part 13

Posted on • Updated on

DP-900 Part 13

Real time analytics

Data Processing : Converting of raw data to meaningful information.

  1. Batch Processing :

Group of stored data, processed as batch, data records are stored before processing.

Example : Credit card billing

  1. Stream Processing :

Data is processed in real time as it occurs. For dynamic data generation, and critical operations.

Example : Stock market change

Batch vs Stream

  1. Batch handles large dataset while stream handles individual record

  2. Batch has access to all datasets, while stream has access to recent data

  3. Batch returns result in few hours, while stream in few second

Architecture of stream processing

  • Event generates some data
  • Data is captured in streaming source for processing
  • Event data is processed, query operators on event data
  • result are stored on output

Real time analytics in Azure

  • Azure stream analytics
  • Spark structured analytics
  • Azure data explorer

Thanks for reading...

Top comments (0)

Timeless DEV post...

Git Concepts I Wish I Knew Years Ago

The most used technology by developers is not Javascript.

It's not Python or HTML.

It hardly even gets mentioned in interviews or listed as a pre-requisite for jobs.

I'm talking about Git and version control of course.

One does not simply learn git