Microsoft Certified: Azure Solutions Architect ExpertDesign data storage solutionsMedium
A telecommunications company has petabytes of call detail records (CDRs) stored in an on-premises Hadoop Distributed File System (HDFS). They need to migrate this data to Azure and perform complex analytical queries using Spark and other big data tools. The solution must support a hierarchical file system, be highly scalable, and cost-effective for long-term storage and analytics. Which Azure data storage solution is best suited for this migration?
- AAzure Blob Storage (Archive Tier)
- BAzure Database for PostgreSQL
- CAzure Data Lake Storage Gen2
- DAzure SQL Database
Show answer & explanationAnswer & explanation
Correct answer: C. Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is designed for big data analytics, offering petabyte-scale storage, a hierarchical file system, and native integration with Spark and other big data tools. It provides a cost-effective solution for storing and analyzing large volumes of unstructured and semi-structured data like CDRs, making it ideal for migrating from HDFS.
Why the other options are wrong
- A. Azure Blob Storage (Archive Tier) is for very long-term, rarely accessed data and does not offer the hierarchical file system or direct integration with big data analytics engines required.
- B. Azure Database for PostgreSQL is a relational database and is not designed for petabyte-scale semi-structured data analytics with big data tools.
- D. Azure SQL Database is a relational database and is not suitable for storing petabytes of semi-structured CDRs from HDFS or for integration with big data analytics tools like Spark.
Azure Data Lake Storage Gen2 (HDFS Integration)
ADLS Gen2 provides a hierarchical namespace that allows it to present object storage as a file system, making it fully compatible with Hadoop Distributed File System (HDFS) semantics and tools like Apache Spark and Hadoop.
- Leverages Azure Blob Storage's scale and cost
- Enables POSIX-compliant file system operations
- Optimized for analytical workloads
Memory trick: ADLS Gen2: HDFS's best 'Gen'etic match in the cloud.