Professional Data EngineerDesigning data processing systemsEasy
A financial institution processes millions of transactions daily. Due to regulatory requirements, all transaction data must be retained for 7 years for auditing and compliance, but only the most recent 90 days of data are actively queried for operational reporting. The remaining historical data is rarely accessed but must be available if needed. The company wants to minimize storage costs while ensuring data availability and compliance. Which BigQuery feature should be primarily used to optimize costs for this scenario?
- ABigQuery reservations
- BBigQuery Data Transfer Service
- CBigQuery BI Engine
- DBigQuery long-term storage
Show answer & explanationAnswer & explanation
Correct answer: D. BigQuery long-term storage
BigQuery's long-term storage automatically applies to data that has not been modified for 90 consecutive days. This feature significantly reduces storage costs (by 50%) for infrequently accessed data while keeping it immediately available for queries, perfectly aligning with the requirement to retain data for 7 years with minimal access after 90 days.
Why the other options are wrong
- A. BigQuery reservations are for query cost optimization (flat-rate pricing), not storage cost optimization for infrequently accessed data.
- B. Data Transfer Service is for moving data into BigQuery, not for managing storage tiers.
- C. BI Engine is for accelerating queries and dashboards, not for storage cost optimization.
BigQuery Long-Term Storage
An automatic storage tier in BigQuery that reduces costs by 50% for data that has not been modified for 90 consecutive days, without impacting query performance.
- Automatic cost reduction.
- Applies after 90 days of no modification.
- Data remains immediately queryable.
- Ideal for infrequently accessed historical data.
Memory trick: 90 days of quiet, BigQuery cuts the storage price in half, quite a delight!