Professional Cloud ArchitectDesign and plan a cloud solution architectureHard
A media company hosts a popular online streaming platform on Google Cloud. They need to analyze user viewing habits in real time to provide personalized content recommendations and dynamically adjust video quality. The solution must handle millions of concurrent users and process petabytes of event data daily with minimal latency. Which architecture should they implement?
- ABatch processing with Dataflow and BigQuery for daily recommendations
- BData warehousing with BigQuery for historical analysis and offline model training
- CRelational database with Cloud SQL and App Engine for user profiles
- DReal-time personalization architecture using Pub/Sub, Dataflow, Bigtable, and AI Platform
Show answer & explanationAnswer & explanation
Correct answer: D. Real-time personalization architecture using Pub/Sub, Dataflow, Bigtable, and AI Platform
For real-time personalization with high-volume, low-latency requirements, an architecture leveraging Pub/Sub for ingestion, Dataflow for real-time processing, Bigtable for low-latency feature serving, and AI Platform for real-time model inference is ideal. This enables immediate recommendations and dynamic adjustments.
Why the other options are wrong
- A. Batch processing is for daily or less frequent updates, not for 'real-time' personalized recommendations and dynamic adjustments.
- B. Data warehousing with BigQuery is excellent for historical analysis and offline model training, but not for serving real-time recommendations and dynamically adjusting content based on immediate user interactions.
- C. A relational database like Cloud SQL is not designed for the scale of 'millions of concurrent users' and 'petabytes of event data' for real-time analytics and serving.
Real-time Personalization Architecture
An architecture designed to deliver immediate, customized user experiences by processing user behavior data in real time, leveraging services for ingestion, stream processing, low-latency feature serving, and model inference.
- Ingestion: Pub/Sub (high-throughput messaging)
- Stream Processing: Dataflow (real-time data transformations)
- Feature Serving: Bigtable (low-latency access to features/profiles)
- Model Inference: AI Platform (real-time prediction service)
Memory trick: Pub/Sub + Dataflow + Bigtable + AI: Instant Personal Touch.