A global IoT company collects sensor data from millions of devices worldwide. This data is ingested into Cloud Pub/Sub and then processed by a Dataflow pipeline. The company needs to optimize costs by ensuring that the Dataflow pipeline only uses the necessary compute resources at any given time, dynamically adjusting to varying data ingestion rates throughout the day. Which Dataflow feature should be configured to achieve this cost optimization?

Why the other options are wrong

• B. Worker disk type affects storage performance and cost, but it does not control the number of compute resources (workers) used by the pipeline, which is the primary driver of compute cost optimization.
• C. Manual scaling requires constant human intervention to adjust resources, which is inefficient and prone to errors, failing the 'dynamically adjusting' requirement.
• D. A fixed number of workers would not dynamically adjust to varying data rates, leading to over-provisioning during low periods and under-provisioning during high periods, incurring unnecessary costs or performance issues.