AWS Certified AI PractitionerFoundation ModelsHard
A company is developing a new customer service chatbot and needs to select a foundation model. The primary constraint is a limited budget for operational expenses, meaning the cost associated with running the model for each user interaction must be minimized. Which factor should the team prioritize when selecting the model?
- AModel's training dataset size
- BModel's pre-training duration
- CAvailability of open-source fine-tuning scripts
- DFoundation model inference cost
Show answer & explanationAnswer & explanation
Correct answer: D. Foundation model inference cost
The primary constraint is a 'limited budget for operational expenses' and minimizing 'cost associated with running the model for each user interaction'. This directly refers to the inference cost, which is the computational expense (CPU/GPU usage, memory) incurred each time the model processes a request. Models vary significantly in their inference cost, with larger or less optimized models being more expensive to run per query.
Why the other options are wrong
- A. Training dataset size impacts model capability but not directly the operational cost per interaction.
- B. Model's pre-training duration affects development cost and model capability, not the ongoing per-interaction operational expense.
- C. Availability of fine-tuning scripts impacts development time, not the recurring operational cost.
Foundation Model Inference Cost
The computational expense (e.g., GPU hours, memory, electricity) incurred each time a foundation model processes an input and generates an output (performs inference).
- Directly impacts operational expenses for AI applications
- Influenced by model size, architecture, and hardware used
- Can be optimized through techniques like quantization and distillation
Memory trick: Each answer given, a coin is spent.