Microsoft Azure Data FundamentalsDescribe how to work with non-relational data on AzureMedium
A data engineer needs to ingest a continuous stream of JSON log data from various applications into Azure for real-time analytics. The data schema is semi-structured and can vary slightly over time. The solution needs to support high-volume ingestion and allow for flexible querying of the JSON data. Which Azure non-relational data service is the most appropriate for storing this data?
- AAzure Cosmos DB (SQL API)
- BAzure Table Storage
- CAzure Data Lake Storage Gen2
- DAzure Blob Storage
Show answer & explanationAnswer & explanation
Correct answer: A. Azure Cosmos DB (SQL API)
Azure Cosmos DB's SQL API is a document database that is excellent for storing semi-structured JSON data. It offers schema flexibility, high-volume ingestion, and powerful querying capabilities over JSON documents, making it ideal for real-time analytics on log data with evolving schemas.
Why the other options are wrong
- B. Azure Table Storage is a key-value store for structured data, not best suited for semi-structured JSON with flexible querying.
- C. Azure Data Lake Storage Gen2 is optimized for big data file storage and batch analytics, not typically for real-time, flexible querying of individual JSON documents in a transactional manner.
- D. Azure Blob Storage is for unstructured objects; while it can store JSON files, it doesn't provide native querying capabilities over the JSON content or transactional support suitable for real-time ingestion and analytics.
Azure Cosmos DB SQL API
A document database API for Azure Cosmos DB, optimized for storing and querying semi-structured JSON data.
- Stores data as JSON documents.
- Supports SQL-like query language (SQL API).
- Offers schema flexibility (schemaless).
- Ideal for web, mobile, gaming, and IoT applications with semi-structured data.
Memory trick: Cosmos is the king for querying JSON data.