Professional Data EngineerEnsuring solution qualityMedium
A gaming company is building a real-time analytics platform to track player engagement and in-game purchases. The platform needs to ingest millions of events per second, process them with very low latency (sub-second), and store them for immediate querying. Data durability and ordering are critical to ensure accurate analytics. You need to select the most appropriate Google Cloud service for event ingestion that meets these requirements. Which service should you choose?
- ABigQuery.
- BCloud Storage.
- CCloud Pub/Sub.
- DCloud SQL.
Show answer & explanationAnswer & explanation
Correct answer: C. Cloud Pub/Sub.
Cloud Pub/Sub is a fully managed, real-time messaging service designed for ingesting large volumes of events (millions per second) with low latency. It provides at-least-once delivery and can ensure message ordering within a topic, making it ideal for real-time event ingestion where durability and ordering are critical.
Why the other options are wrong
- A. BigQuery is a data warehouse for analytics, not an event ingestion service. While it supports streaming inserts, Pub/Sub is the primary ingestion layer for high-volume, real-time events.
- B. Cloud Storage is an object storage service, suitable for batch ingestion or storing raw files, but not for real-time, low-latency event streaming.
- D. Cloud SQL is a relational database, not designed for ingesting millions of events per second with low latency in a streaming architecture.
Cloud Pub/Sub for Event Ingestion
A real-time messaging service on Google Cloud for scalable and low-latency event ingestion and delivery.
- High throughput (millions/second).
- Low latency (sub-second).
- At-least-once delivery, message ordering.
- Fully managed and scalable.
Memory trick: Pub/Sub is like the post office for your real-time events: it handles millions of messages, quickly and reliably, ensuring they get to their destination in order.