Professional Data EngineerBuilding and operationalizing data processing systemsEasy
A global media company needs to process millions of video frames per second for real-time content moderation. Each frame needs to be analyzed by a machine learning model, and the processing must be highly scalable and fault-tolerant. The processing logic is complex and requires custom code. Which Google Cloud service is best suited for this task?
- ACloud SQL
- BBigQuery
- CDataflow
- DCloud Spanner
Show answer & explanationAnswer & explanation
Correct answer: C. Dataflow
Dataflow is a fully managed service for executing Apache Beam pipelines. It provides autoscaling and fault tolerance, making it ideal for large-scale, real-time data processing with custom logic.
Why the other options are wrong
- A. Cloud SQL is a managed relational database service, not suitable for high-throughput stream processing.
- B. BigQuery is a serverless data warehouse for analytics, not primarily for real-time custom processing of individual data elements.
- D. Cloud Spanner is a globally distributed relational database, not a data processing engine.
Google Cloud Dataflow
A fully managed service for executing Apache Beam pipelines, enabling reliable and scalable processing of both batch and streaming data.
- Supports both batch and stream processing
- Automatically scales resources up and down
- Provides exactly-once processing guarantees
Memory trick: Dataflow: The flexible river for your data.