Professional Data EngineerManaging and securing dataMedium

A telecommunications company uses Pub/Sub for ingesting real-time call detail records (CDRs). Due to regulatory compliance, these records must be retained for exactly 30 days and then automatically purged. They also need to ensure that if a subscriber is temporarily unavailable, messages are not lost and can be redelivered later. How should you configure the Pub/Sub topic and subscription to meet these requirements?

  1. AConfigure a topic with message retention duration of 30 days and a dead-letter topic on the subscription.
  2. BConfigure a subscription with an acknowledgment deadline of 30 days and a retry policy.
  3. CConfigure a subscription with a message retention duration of 30 days and a message ordering enabled.
  4. DConfigure a subscription with a message retention duration of 30 days and a dead-letter topic.
Show answer & explanation

Correct answer: D. Configure a subscription with a message retention duration of 30 days and a dead-letter topic.

Setting the subscription's message retention duration to 30 days ensures that messages are automatically purged after that period. A dead-letter topic ensures that messages that cannot be successfully processed after a certain number of retries (e.g., due to a temporarily unavailable subscriber) are moved to a separate topic instead of being lost, allowing for later analysis or reprocessing.

Why the other options are wrong

  • A. Topic message retention allows for messages to be replayed, but the question specifies purging after 30 days and redelivery for individual subscribers, which are handled at the subscription level. A dead-letter topic is correctly placed on the subscription.
  • B. An acknowledgment deadline defines how long a subscriber has to acknowledge a message; 30 days is excessively long and not for retention. A retry policy is part of handling failed messages, but a dead-letter topic is the mechanism to prevent loss and allow for later redelivery/analysis for persistent failures.
  • C. Message ordering ensures messages are delivered in the order they were published but does not address retention or redelivery for failed processing.

Pub/Sub Message Retention & Dead-Letter Topics

Pub/Sub subscriptions allow configuring message retention duration to automatically purge messages after a specified time. Dead-letter topics provide a mechanism to catch messages that cannot be successfully processed, preventing loss and enabling debugging or reprocessing.

  • Message retention is configured at the subscription level (up to 7 days, or 31 days with topic retention).
  • Dead-letter topics are configured on subscriptions.
  • Messages sent to a dead-letter topic are not lost.
  • Useful for compliance and robust message processing.

Memory trick: Retain the messages, but don't let them die, use a dead-letter, for failures high!

More Managing and securing data questions