Microsoft Azure Data FundamentalsDescribe how to work with non-relational data on AzureMedium
An analytics team needs to store raw sensor data from IoT devices for long-term analysis. The data arrives in varying formats and volumes, and the team intends to use various big data processing frameworks like Apache Spark and Hadoop for analysis. Which Azure non-relational data service is specifically designed to support these requirements for large-scale analytics?
- AAzure Blob Storage (General-purpose v2)
- BAzure Data Lake Storage Gen2
- CAzure Table Storage
- DAzure Cosmos DB
Show answer & explanationAnswer & explanation
Correct answer: B. Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is a highly scalable and cost-effective data lake solution built on Azure Blob Storage. It provides a hierarchical namespace, making it compatible with Hadoop Distributed File System (HDFS) and optimized for big data analytics workloads like Spark and Hadoop, handling diverse data formats and volumes.
Why the other options are wrong
- A. While Azure Blob Storage can store large data, Data Lake Storage Gen2 adds a hierarchical namespace and optimizations specifically for big data analytics frameworks, making it a better fit for this scenario.
- C. Azure Table Storage is a key-value store, not suitable for storing raw, varied-format sensor data for big data analytics.
- D. Azure Cosmos DB is a transactional NoSQL database, not optimized for raw, diverse-format, large-scale analytics with Spark/Hadoop.
Azure Data Lake Storage Gen2
A highly scalable and secure data lake solution for big data analytics workloads, built on Azure Blob Storage.
- HDFS compatible (hierarchical namespace).
- Optimized for Apache Spark, Hadoop, and other big data frameworks.
- Supports petabyte-scale data with high throughput.
- Offers enterprise-grade security features.
Memory trick: Data Lake is the big pond for all your analytics fish.