Professional Data EngineerDesigning data processing systemsMedium
A global IoT company collects sensor data from millions of devices worldwide. This data is critical for operational monitoring and predictive maintenance. The data arrives as high-volume, low-latency streams and needs to be stored in a time-series optimized database that can handle petabytes of data with very fast writes and efficient queries across time ranges. The solution must also support high availability and replication across multiple regions to ensure business continuity. Which Google Cloud service is best suited for storing this IoT sensor data?
- ABigQuery
- BCloud Bigtable
- CCloud SQL
- DCloud Spanner
Show answer & explanationAnswer & explanation
Correct answer: B. Cloud Bigtable
Cloud Bigtable is a fully managed, scalable NoSQL wide-column database service designed for large analytical and operational workloads, including IoT time-series data. It excels at high-throughput, low-latency reads and writes, making it ideal for ingesting millions of sensor readings per second and performing efficient queries across time ranges. It also supports replication for high availability.
Why the other options are wrong
- A. BigQuery is a data warehouse optimized for analytical queries, but its ingestion pattern and cost model are not ideal for continuous, high-volume, low-latency writes of individual sensor readings, especially for operational monitoring.
- C. Cloud SQL is a relational database not optimized for petabyte-scale time-series data with high write throughput and low-latency queries across time ranges.
- D. Cloud Spanner is a globally distributed relational database, offering strong consistency and high availability, but it's generally more expensive and optimized for transactional workloads, not typically the most cost-effective choice for raw IoT time-series data storage at petabyte scale.
Cloud Bigtable for Time-Series
Cloud Bigtable is a NoSQL wide-column database optimized for large analytical and operational workloads, particularly well-suited for high-throughput, low-latency time-series data like IoT sensor readings due to its efficient storage and retrieval mechanisms.
- Handles millions of writes/reads per second.
- Designed for petabyte-scale data.
- Low-latency access, ideal for time-series and operational data.
- Supports multi-region replication for high availability.
Memory trick: BigTable stores Big Time-Series.