Interview Question
A RAG application has good latency but poor answer quality. How do you isolate the problem?
What a Strong Answer Should Cover
- Measure retrieval separately
- Inspect chunking and metadata
- Evaluate ranking and context construction
- Test model behavior independently
- Use a representative evaluation set
Common Mistakes
- Jumping to a technology before clarifying the problem
- Explaining the solution without the reasoning or trade-offs
- Omitting verification, failure handling, or prevention
Interviewer Follow-ups
- How would you prove retrieval is the problem?
- Which evaluation set would you use?
- What change would you test first?
What the interviewer is testing
- Problem isolation and evidence-based reasoning
- Safe mitigation and verification
- Prevention and operational learning
Real-World Sample Answer
A strong real-world response should connect requirements to engineering decisions, explain why the chosen approach fits the constraints, and make failure, security, cost, observability, and verification explicit.