Professional Data EngineerBuilding and operationalizing data processing systemsMedium
A data engineering team is building a real-time recommendation engine. They need to store user interaction data (e.g., clicks, views, purchases) and serve personalized recommendations with extremely low latency (sub-10ms) to millions of concurrent users. The data model is relatively simple, consisting of key-value pairs and wide-column structures. High write throughput and read throughput are critical. Which Google Cloud service is best suited for this operational database requirement?
- ACloud Bigtable
- BBigQuery
- CCloud SQL
- DCloud Spanner
Show answer & explanationAnswer & explanation
Correct answer: A. Cloud Bigtable
Cloud Bigtable is a fully managed NoSQL wide-column database designed for high throughput and low-latency access to large datasets, making it ideal for real-time recommendation engines and other operational analytics workloads with millions of concurrent users.
Why the other options are wrong
- B. BigQuery is an analytical data warehouse optimized for large-scale batch queries, not for low-latency operational lookups for individual recommendations.
- C. Cloud SQL is a relational database and cannot provide the extreme scale, high throughput, and low latency required for millions of concurrent users in a real-time recommendation engine.
- D. Cloud Spanner is a globally distributed relational database, offering strong consistency and horizontal scalability, but Bigtable is generally more cost-effective and performant for pure key-value/wide-column NoSQL use cases at this scale.
Cloud Bigtable for Real-time Analytics
Google Cloud's fully managed NoSQL wide-column database, optimized for large analytical and operational workloads requiring high throughput and low-latency access.
- High read/write throughput (millions of ops/sec)
- Sub-10ms latency for point reads
- Ideal for time-series, IoT, and operational analytics
Memory trick: Bigtable handles big data fast, with wide columns for real-time blast.