Microsoft Azure Data FundamentalsDescribe how to work with non-relational data on AzureHard
A data analytics team needs to store petabytes of raw sensor data from IoT devices for long-term analysis. The data arrives in various formats (CSV, JSON, Avro) and will be processed by big data analytics engines like Apache Spark. The storage solution must support hierarchical namespaces, fine-grained access control, and be optimized for analytical workloads. Which Azure Storage service is the most suitable?
- AAzure Blob Storage
- BAzure Data Lake Storage Gen2
- CAzure Cosmos DB
- DAzure Table Storage
Show answer & explanationAnswer & explanation
Correct answer: B. Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is built on Azure Blob Storage but adds a hierarchical namespace and optimized drivers for big data analytics, making it ideal for petabyte-scale data, various formats, and integration with Spark.
Why the other options are wrong
- A. Azure Blob Storage can store large data, but lacks the hierarchical namespace and analytics optimizations of ADLS Gen2.
- C. Azure Cosmos DB is a transactional NoSQL database, not designed for raw data lake storage and big data analytics.
- D. Azure Table Storage is a key-value store, not suitable for petabyte-scale raw data in various formats for analytics.
Azure Data Lake Storage Gen2
A set of capabilities dedicated to big data analytics, built on Azure Blob Storage, combining the scalability and cost-effectiveness of object storage with the features of a data lake.
- Hierarchical namespace for folder and file-level security.
- Optimized for analytical workloads (Spark, Hadoop).
- Petabyte-scale storage for various data formats.
- Supports fine-grained access control (ACLs).
Memory trick: Data Lake Gen2 is the grand lake for all your big data analysis.