Microsoft Certified: Azure Solutions Architect ExpertDesign infrastructure solutionsMedium
A consulting firm is designing a new solution for a client that involves processing large volumes of semi-structured data (e.g., JSON logs, CSV files) for analytical purposes. The data will be ingested from various sources, stored for long-term retention, and then analyzed using big data analytics tools like Apache Spark. The solution requires a highly scalable, cost-effective storage service that supports hierarchical namespaces and fine-grained access control. Which Azure service should be used?
- AAzure Table Storage
- BAzure Blob Storage Standard
- CAzure Data Lake Storage Gen2
- DAzure SQL Data Warehouse (Synapse Analytics)
Show answer & explanationAnswer & explanation
Correct answer: C. Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is specifically optimized for big data analytics workloads. It combines the scalability and cost-effectiveness of Azure Blob Storage with a hierarchical namespace and file system semantics, making it ideal for storing petabytes of semi-structured data and integrating seamlessly with analytics engines like Apache Spark, while also providing fine-grained access control.
Why the other options are wrong
- A. Azure Table Storage is a NoSQL key-value store, not suitable for storing large volumes of semi-structured files for big data analytics.
- B. Azure Blob Storage Standard is object storage. While scalable and cost-effective, it lacks the hierarchical namespace and optimized performance for big data analytics tools that ADLS Gen2 offers.
- D. Azure SQL Data Warehouse (now Synapse Analytics SQL pools) is a data warehousing service, not a primary storage layer for raw semi-structured data in a data lake.
Azure Data Lake Storage Gen2
A highly scalable and secure data lake solution built on Azure Blob Storage, optimized for big data analytics workloads.
- Hierarchical namespace for folder/file organization
- Exabyte-scale storage, cost-effective
- Optimized for Apache Spark and other big data engines
Memory trick: Data Lake Gen2: Your 'Lake' of data, 'Gen'erally for analytics.