A global e-commerce company wants to build a new recommendation engine. The engine needs to process user clickstream data in real-time, enrich it with user profile information from a low-latency NoSQL database, and then feed the enriched data to a machine learning model for immediate prediction. The solution must handle millions of events per second and maintain sub-100ms latency for enrichment. Which Google Cloud services should be chosen for the low-latency NoSQL database and the real-time stream processing?
- ACloud SQL for user profiles and Dataflow for stream processing.
- BBigtable for user profiles and Dataflow in streaming mode for stream processing.
- CCloud Spanner for user profiles and Dataproc for stream processing.
- DFirestore for user profiles and Cloud Functions for stream processing.
Show answer & explanationAnswer & explanation
Correct answer: B. Bigtable for user profiles and Dataflow in streaming mode for stream processing.
Bigtable is a high-performance, fully managed NoSQL database service suitable for large analytical and operational workloads requiring low-latency reads and writes, making it ideal for user profiles. Dataflow in streaming mode is designed for real-time processing of high-volume event streams, offering auto-scaling and exactly-once processing, perfect for enriching clickstream data.
Why the other options are wrong
- A. Cloud SQL is a relational database with higher latency and less scalability for millions of reads/writes per second compared to NoSQL options for this use case.
- C. Cloud Spanner is a globally distributed relational database, not a NoSQL solution. Dataproc is primarily for batch processing with Hadoop/Spark, not optimal for real-time stream processing with sub-100ms latency.
- D. Firestore is a NoSQL document database, but Bigtable is generally better suited for the high-throughput, low-latency requirements of a massive user profile store. Cloud Functions are event-driven and not designed for continuous stream processing at this scale.
Bigtable for Low-Latency NoSQL
Cloud Bigtable is a fully managed, scalable NoSQL wide-column database service designed for large analytical and operational workloads, offering very low latency for high-throughput reads and writes, making it ideal for real-time applications.
- High throughput and low latency for petabyte-scale data.
- Ideal for time-series data, marketing data, financial data, and IoT data.
- Supports open-source HBase API.
- Fully managed with automatic scaling and replication.
Memory trick: BigTable + Dataflow = Fast Recommendations.