Microsoft Certified: Azure AI Engineer AssociateImplement generative AI solutionsHard
A gaming company is developing an interactive storytelling game using Azure OpenAI Service. They want to ensure that player choices and their impact on the narrative are dynamically and consistently remembered across long conversations, allowing for complex branching storylines. However, they observe that the model sometimes 'forgets' earlier plot points or character actions. Which approach to managing conversational context is most suitable for this scenario?
- ASetting a high 'frequency_penalty' to reduce redundant plot elements.
- BLimiting Max_tokens to keep conversations concise.
- CImplementing a sliding window approach to manage the conversation history.
- DFine-tuning the model with every new player interaction.
Show answer & explanationAnswer & explanation
Correct answer: C. Implementing a sliding window approach to manage the conversation history.
A sliding window approach dynamically maintains a relevant portion of the conversation history within the LLM's context window. This allows the model to 'remember' recent interactions and plot points without exceeding token limits, which is crucial for long, interactive narratives.
Why the other options are wrong
- A. Frequency penalty reduces repetition, but doesn't help the model remember past unique events or plot points.
- B. Limiting Max_tokens would truncate the conversation history, exacerbating the 'forgetting' problem.
- D. Fine-tuning with every interaction is impractical, extremely expensive, and doesn't solve the real-time context management problem during an ongoing conversation.
Sliding Window Context Management
A technique for managing long conversation histories with LLMs by only including the most recent 'N' tokens or turns in the prompt.
- Addresses the LLM's fixed context window limitation.
- Keeps the conversation relevant to recent interactions.
- Can be combined with other techniques like summarization for critical information.
Memory trick: Slide the Window for Long Story Memory: 'Sliding window' helps your story's 'memory' last longer.