KAUSTUBHSHARMA.COM

Interview Question Bank

AI & ML Interview Questions

AI, GenAI, RAG, ML, and MLOps interview questions.

Click a question to reveal its answer and related guidance.

Q1Design a production AI agent that can call business tools without becoming an uncontrolled automation risk.

What a Strong Answer Should Cover

  • Separate model reasoning from tools
  • Least-privilege tool access
  • Argument validation and policy checks
  • Step and cost limits
  • Observability and human escalation

Common Mistakes

  • Jumping to a technology before clarifying the problem
  • Explaining the solution without the reasoning or trade-offs
  • Omitting verification, failure handling, or prevention

Interviewer Follow-ups

  • What tools should the agent never be allowed to call?
  • How do you constrain a runaway workflow?
  • Where should a human approve actions?

What the interviewer is testing

  • Requirements and constraints
  • Architecture and trade-offs
  • Reliability, security, cost, and operations

Real-World Sample Answer

A strong real-world response should connect requirements to engineering decisions, explain why the chosen approach fits the constraints, and make failure, security, cost, observability, and verification explicit.

What the interviewer is testing

  • Security boundaries
  • Tool authorization
  • Safe failure

Real-World Sample Answer

I would approach “How would you keep an AI agent from performing an unsafe tool action?” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “how would you keep an ai agent from performing an unsafe tool action?”, i would first separate model reasoning from tool permissions. then i would apply least privilege to every tool and validate arguments and policy before execution. i would also use step, cost, and time limits. finally, i would add observability and human approval for sensitive actions. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “How would you keep an AI agent from performing an unsafe tool action?”, I would first separate model reasoning from tool permissions. Then I would apply least privilege to every tool and validate arguments and policy before execution. I would also use step, cost, and time limits. Finally, I would add observability and human approval for sensitive actions. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Trusting the model to self-police
  • Giving broad tool permissions
  • No action-level audit trail

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?

What the interviewer is testing

  • Core AI/GenAI concept recognition
  • Ability to explain the concept accurately and simply

Real-World Sample Answer

['A language model for reasoning', 'Tools or actions for interacting with external systems', 'A requirement to avoid all external data', 'A rule that every task must use the same fixed answer']

Common Mistakes

  • Memorizing a definition without being able to explain when the concept is useful
  • Naming a technology without explaining the engineering purpose

Use the connected AI and GenAI learning resources on the site to go deeper after answering.

What the interviewer is testing

  • Core AI/GenAI concept recognition
  • Ability to explain the concept accurately and simply

Real-World Sample Answer

Agentic AI systems can reason about a goal and take actions toward it, often combining language models, tools, planning, and feedback loops.

Common Mistakes

  • Memorizing a definition without being able to explain when the concept is useful
  • Naming a technology without explaining the engineering purpose

Use the connected AI and GenAI learning resources on the site to go deeper after answering.

What the interviewer is testing

  • Core AI/GenAI concept recognition
  • Ability to explain the concept accurately and simply

Real-World Sample Answer

Grounding the system with relevant source material and evaluating responses against representative tasks are both important quality practices.

Common Mistakes

  • Memorizing a definition without being able to explain when the concept is useful
  • Naming a technology without explaining the engineering purpose

Use the connected AI and GenAI learning resources on the site to go deeper after answering.

What the interviewer is testing

  • End-to-end architecture
  • Authorization
  • Evaluation
  • Operations

Real-World Sample Answer

I would approach “Design a production RAG platform for an enterprise knowledge base with strict access controls.” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “design a production rag platform for an enterprise knowledge base with strict access controls.”, i would first design ingestion and document lifecycle. then i would separate retrieval, ranking, and context construction and enforce document-level authorization. i would also define evaluation and grounding signals. finally, i would include observability, cost controls, and fallback behavior. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “Design a production RAG platform for an enterprise knowledge base with strict access controls.”, I would first design ingestion and document lifecycle. Then I would separate retrieval, ranking, and context construction and enforce document-level authorization. I would also define evaluation and grounding signals. Finally, I would include observability, cost controls, and fallback behavior. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Treating the vector database as the whole architecture
  • Applying authorization only after retrieval
  • No evaluation strategy

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?

What the interviewer is testing

  • Quality-cost trade-offs
  • Measurement
  • Workload optimization

Real-World Sample Answer

I would approach “How would you reduce the cost of a high-volume LLM application without sacrificing the required quality?” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “how would you reduce the cost of a high-volume llm application without sacrificing the required quality?”, i would first define quality and latency budgets first. then i would measure cost by workload and call type and reduce unnecessary context and calls. i would also use model selection or routing where appropriate. finally, i would validate every optimization against the quality target. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “How would you reduce the cost of a high-volume LLM application without sacrificing the required quality?”, I would first define quality and latency budgets first. Then I would measure cost by workload and call type and reduce unnecessary context and calls. I would also use model selection or routing where appropriate. Finally, I would validate every optimization against the quality target. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Choosing the cheapest model without a quality target
  • Optimizing token count without measuring outcome
  • Ignoring caching and repeated work

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?
  • Use the Interview Preparation learning paths and related site content to deepen any topic you could not confidently explain.

What the interviewer is testing

  • Production ML diagnosis
  • Data quality
  • Model comparison

Real-World Sample Answer

I would approach “A model's production quality dropped even though the training pipeline did not change. What would you investigate?” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “a model's production quality dropped even though the training pipeline did not change. what would you investigate?”, i would first check feature and data drift. then i would investigate label quality and delayed feedback and look for training-serving skew. i would also compare model and data versions. finally, i would use monitoring and rollback when the failure signal is clear. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “A model's production quality dropped even though the training pipeline did not change. What would you investigate?”, I would first check feature and data drift. Then I would investigate label quality and delayed feedback and look for training-serving skew. I would also compare model and data versions. Finally, I would use monitoring and rollback when the failure signal is clear. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Assuming the model is unchanged so behavior should be unchanged
  • Ignoring data pipelines
  • No rollback plan

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?

What a Strong Answer Should Cover

  • Trace data and model artifacts
  • Automate evaluation gates
  • Use a model registry
  • Monitor production behavior
  • Support rollback

Common Mistakes

  • Jumping to a technology before clarifying the problem
  • Explaining the solution without the reasoning or trade-offs
  • Omitting verification, failure handling, or prevention

Interviewer Follow-ups

  • What makes a model eligible for promotion?
  • How would you detect a bad model after release?
  • How would you roll back safely?

What the interviewer is testing

  • Requirements and constraints
  • Architecture and trade-offs
  • Reliability, security, cost, and operations

Real-World Sample Answer

A strong real-world response should connect requirements to engineering decisions, explain why the chosen approach fits the constraints, and make failure, security, cost, observability, and verification explicit.

What the interviewer is testing

  • Evaluation design
  • Artifact lineage
  • Release safety

Real-World Sample Answer

I would approach “What gates should a model pass before it is promoted to production?” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “what gates should a model pass before it is promoted to production?”, i would first define task-specific quality thresholds. then i would track data and model artifacts and compare against the current production model. i would also check safety, latency, and resource constraints. finally, i would define promotion and rollback conditions. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “What gates should a model pass before it is promoted to production?”, I would first define task-specific quality thresholds. Then I would track data and model artifacts and compare against the current production model. I would also check safety, latency, and resource constraints. Finally, I would define promotion and rollback conditions. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Using one generic accuracy metric
  • No lineage
  • Promoting based only on offline performance

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?
  • Use the Interview Preparation learning paths and related site content to deepen any topic you could not confidently explain.

What a Strong Answer Should Cover

  • Define quality and latency budgets
  • Measure cost by workload
  • Use routing or model selection
  • Reduce unnecessary context and calls
  • Validate changes with evaluation

Common Mistakes

  • Jumping to a technology before clarifying the problem
  • Explaining the solution without the reasoning or trade-offs
  • Omitting verification, failure handling, or prevention

Interviewer Follow-ups

  • What quality target must remain fixed?
  • Where is the largest avoidable cost?
  • How would you validate that cheaper is still good enough?

What the interviewer is testing

  • Requirements and constraints
  • Architecture and trade-offs
  • Reliability, security, cost, and operations

Real-World Sample Answer

A strong real-world response should connect requirements to engineering decisions, explain why the chosen approach fits the constraints, and make failure, security, cost, observability, and verification explicit.

What the interviewer is testing

  • Core AI/GenAI concept recognition
  • Ability to explain the concept accurately and simply

Real-World Sample Answer

Retrieval-Augmented Generation (RAG) retrieves relevant context before generation so an LLM can use external or current knowledge at inference time.

Common Mistakes

  • Memorizing a definition without being able to explain when the concept is useful
  • Naming a technology without explaining the engineering purpose

Use the connected AI and GenAI learning resources on the site to go deeper after answering.

What the interviewer is testing

  • Layered diagnosis
  • Evidence-based evaluation
  • Fallback design

Real-World Sample Answer

I would approach “A model gives confident answers that are not supported by retrieved documents. How would you troubleshoot it?” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “a model gives confident answers that are not supported by retrieved documents. how would you troubleshoot it?”, i would first separate retrieval quality from prompt/context construction. then i would check whether retrieved evidence actually supports the answer and evaluate model behavior on a representative set. i would also define refusal or fallback behavior. finally, i would instrument retrieval, context, and generation separately. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “A model gives confident answers that are not supported by retrieved documents. How would you troubleshoot it?”, I would first separate retrieval quality from prompt/context construction. Then I would check whether retrieved evidence actually supports the answer and evaluate model behavior on a representative set. I would also define refusal or fallback behavior. Finally, I would instrument retrieval, context, and generation separately. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Blaming the model immediately
  • Using only anecdotal examples
  • No measurable grounding target

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?

What a Strong Answer Should Cover

  • Measure retrieval separately
  • Inspect chunking and metadata
  • Evaluate ranking and context construction
  • Test model behavior independently
  • Use a representative evaluation set

Common Mistakes

  • Jumping to a technology before clarifying the problem
  • Explaining the solution without the reasoning or trade-offs
  • Omitting verification, failure handling, or prevention

Interviewer Follow-ups

  • How would you prove retrieval is the problem?
  • Which evaluation set would you use?
  • What change would you test first?

What the interviewer is testing

  • Problem isolation and evidence-based reasoning
  • Safe mitigation and verification
  • Prevention and operational learning

Real-World Sample Answer

A strong real-world response should connect requirements to engineering decisions, explain why the chosen approach fits the constraints, and make failure, security, cost, observability, and verification explicit.

What a Strong Answer Should Cover

  • Clear retrieval and generation flow
  • Scalable ingestion and serving path
  • Security and tenant isolation
  • Cost controls and observability
  • Failure handling and reliability trade-offs

Common Mistakes

  • Jumping to a technology before clarifying the problem
  • Explaining the solution without the reasoning or trade-offs
  • Omitting verification, failure handling, or prevention

Interviewer Follow-ups

  • What would you clarify before choosing components?
  • How would you evaluate retrieval and generation separately?
  • What trade-off would you revisit at 10x traffic?

What the interviewer is testing

  • Requirements and constraints
  • Architecture and trade-offs
  • Reliability, security, cost, and operations

Real-World Sample Answer

A strong real-world response should connect requirements to engineering decisions, explain why the chosen approach fits the constraints, and make failure, security, cost, observability, and verification explicit.

What the interviewer is testing

  • Evaluation design
  • Retrieval isolation
  • Grounding reasoning

Real-World Sample Answer

I would approach “A RAG application has good latency but poor answer quality. How do you isolate the problem?” by clarifying the requirements first, then using these considerations: in a real interview, i would not jump straight to a technology choice. for “a rag application has good latency but poor answer quality. how do you isolate the problem?”, i would first measure retrieval separately from generation. then i would inspect chunking, metadata, ranking, and context construction and test model behavior independently. i would also use a representative evaluation set. finally, i would add grounding and fallback checks. i would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.. I would state my assumptions and defend the trade-offs rather than presenting the choice as universally correct.

What a Strong Answer Should Cover

In a real interview, I would not jump straight to a technology choice. For “A RAG application has good latency but poor answer quality. How do you isolate the problem?”, I would first measure retrieval separately from generation. Then I would inspect chunking, metadata, ranking, and context construction and test model behavior independently. I would also use a representative evaluation set. Finally, I would add grounding and fallback checks. I would make the assumptions explicit and explain what evidence or production signals would make me revisit the decision.

Common Mistakes

  • Changing models before proving the retrieval problem
  • Evaluating only final answers
  • Ignoring evaluation-set quality

Interviewer Follow-ups

  • What assumptions would you clarify before committing to the design?
  • What changes if the scale, reliability target, security requirement, or budget changes?
  • What is the biggest failure mode in your proposed approach?

What the interviewer is testing

  • Production release safety
  • Model-specific validation
  • Monitoring and rollback reasoning
  • Risk management

Real-World Sample Answer

I would validate the model offline first against a representative evaluation set and define the business and operational thresholds that must hold. Then I would release it gradually, such as through shadow traffic or a canary, while comparing quality, latency, errors, resource use, and business outcomes against the current model. I would define an explicit failure signal and automatic or rapid rollback condition before increasing traffic. After full rollout, I would continue monitoring for drift and delayed feedback and keep the previous model available for safe recovery.

What a Strong Answer Should Cover

  • Offline evaluation and release gates
  • Shadow or canary rollout before broad exposure
  • Comparison against the current production model
  • Quality, latency, error, and resource monitoring
  • Explicit rollback signal and condition
  • Post-release drift and feedback monitoring

Common Mistakes

  • Treating a successful offline score as enough
  • Releasing to 100% of traffic immediately
  • Monitoring infrastructure but not model quality
  • Saying “rollback if it is bad” without defining a signal

Interviewer Follow-ups

  • Which metrics would block the rollout?
  • How would you handle delayed labels or slow business feedback?
  • What would you do if the new model improves quality but increases latency or cost?