CompTIA Data+ (DA0-002)Data Concepts and EnvironmentsMedium
A data engineer is designing a data platform to store petabytes of raw, multi-structured data from various sources including sensor logs, social media feeds, and customer interaction data. The data needs to be stored as-is, without a predefined schema, and made available for future analytical processing, machine learning, and ad-hoc querying by data scientists. Which data environment is BEST suited for this requirement?
- AData Mart
- BData Lake
- CData Warehouse
- DOperational Data Store (ODS)
Show answer & explanationAnswer & explanation
Correct answer: B. Data Lake
A data lake is designed to store vast amounts of raw data in its native format, regardless of structure (structured, semi-structured, unstructured). It supports schema-on-read, allowing data to be flexible and processed for various analytical needs, including machine learning and ad-hoc querying.
Why the other options are wrong
- A. A data mart is a subset of a data warehouse, focused on a specific business area, not for raw, multi-structured data.
- C. A data warehouse stores structured, cleaned, and integrated data, not raw, multi-structured data without a predefined schema.
- D. An ODS is for operational reporting and near-real-time data, not for long-term storage of raw, multi-structured data at petabyte scale.
Data Lake
A centralized repository that allows you to store all your structured and unstructured data at any scale. It stores data in its native format, and you don't have to structure your data before storing it.
- Stores raw data in native format.
- Supports structured, semi-structured, and unstructured data.
- Schema-on-read: schema defined at query time.
- Cost-effective for large volumes of data.
Memory trick: Lake is raw, warehouse is refined, mart is specific, ODS is operational.