A consulting firm is designing a new solution for a client that involves processing large amounts of unstructured data (e.g., documents, images, videos) for machine learning model training and inference. The data needs to be highly available, durable, and accessible from various compute services (e.g., Azure Machine Learning, Azure Databricks). The solution requires petabyte-scale storage with hierarchical namespace capabilities and POSIX-compliant access. Which Azure storage solution should be recommended?
- AAzure Data Lake Storage Gen2
- BAzure Blob Storage (General-purpose v2)
- CAzure Files Premium
- DAzure SQL Database
Show answer & explanationAnswer & explanation
Correct answer: A. Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is specifically designed for big data analytics workloads. It combines the scalability and cost-effectiveness of Azure Blob Storage with a hierarchical namespace and POSIX-compliant access control, making it ideal for storing petabytes of unstructured data for machine learning and big data processing.
Why the other options are wrong
- B. Azure Blob Storage (General-purpose v2) provides object storage but lacks the hierarchical namespace and POSIX-compliant access needed for optimal big data analytics performance and compatibility.
- C. Azure Files Premium provides high-performance SMB/NFS file shares, but it's not optimized for petabyte-scale unstructured data lakes and big data analytics workloads.
- D. Azure SQL Database is a relational database and completely unsuitable for storing large amounts of unstructured data for machine learning.
Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is a highly scalable and secure data lake solution built on Azure Blob Storage. It combines the best features of Blob Storage with a hierarchical file system, making it optimized for big data analytics workloads and machine learning.
- Petabyte-scale storage for big data
- Hierarchical namespace (HNS) for file system semantics
- POSIX-compliant access control (ACLs)
Memory trick: Data Lake Gen2 is the deep, organized ocean for your big data, perfect for training smart machines.