Microsoft Azure Data FundamentalsDescribe core data conceptsMedium
A data architect is designing a system for an autonomous vehicle company. The system needs to ingest and process vast amounts of sensor data (Lidar, Radar, Camera) in real-time, performing immediate analysis to detect obstacles and make navigation decisions. Which data processing option is most appropriate for this scenario?
- AOnline Analytical Processing (OLAP)
- BStream Processing
- COnline Transaction Processing (OLTP)
- DBatch Processing
Show answer & explanationAnswer & explanation
Correct answer: B. Stream Processing
The scenario explicitly states 'real-time' ingestion and 'immediate analysis' for 'navigation decisions' based on continuous sensor data. This perfectly aligns with the capabilities of stream processing, which is designed for continuous data flows requiring instantaneous insights and actions.
Why the other options are wrong
- A. OLAP is for complex analytical queries on historical data, focused on insights over time, not real-time actions.
- C. OLTP is for transactional workloads, focusing on individual, immediate data modifications, not continuous analytical streams.
- D. Batch Processing is for large volumes of data processed periodically, not for real-time decisions.
Stream Processing
A data processing paradigm designed to process continuous, unbounded streams of data in real-time, as the data arrives.
- Enables immediate insights and actions.
- Suitable for IoT, financial trading, fraud detection.
- Data is processed in motion, not at rest.
Memory trick: Stream ahead, Batch behind: Your data's journey needs the right engine.