Data Processing : Converting of raw data to meaningful information.
- Batch Processing :
Group of stored data, processed as batch, data records are stored before processing.
Example : Credit card billing
- Stream Processing :
Data is processed in real time as it occurs. For dynamic data generation, and critical operations.
Example : Stock market change
Batch vs Stream
Batch handles large dataset while stream handles individual record
Batch has access to all datasets, while stream has access to recent data
Batch returns result in few hours, while stream in few second
- Event generates some data
- Data is captured in streaming source for processing
- Event data is processed, query operators on event data
- result are stored on output
- Azure stream analytics
- Spark structured analytics
- Azure data explorer
Thanks for reading...