Capstone: design an LLM application, under interview conditions

Forty-five minutes, one sentence

The interviewer smiles and says it: “Design a customer-support AI for a mid-size bank.” Whiteboard marker in your hand. Everything from this guide is relevant — prompts, retrieval, chunking, eval, budgets, agents, the fine-tuning question — and that’s precisely the trap: you know forty-five minutes of material per chapter, and you have forty-five minutes total.

Predict: what do you spend the first five minutes on — and what should the interviewer hear before any architecture word like “RAG” or “agent” leaves your mouth?

The five steps are this guide, in order

The framework that drives the conversation — and the chapters that armed each step:

1. Problem framing (~5 min) — inputs, outputs, scale, latency SLO, the feared failure, success metrics. Chapter 1’s prompt-as-spec instinct, applied to the whole system: pin the degrees of freedom before designing.
2. High-level architecture (~10 min) — the boxes and the data flow: router → retrieval (Ch 2–3) → generation with grounding (Ch 4) → tools where actions are needed (Ch 8) → validation → response. Name what’s AI and what’s ordinary infrastructure.
3. Component deep dive (~15 min) — pick the 1–2 components where the risk lives and go deep: the retrieval stack and its upgrades (Ch 6), the context budget (Ch 4, 7), the agent loop’s guards (Ch 8), the validation layer. Estimates beat adjectives: tokens, milliseconds, dollars (Ch 7).
4. Trade-offs & failure modes (~8 min) — the genuine tensions, plus “what happens when the model is wrong?” answered with designed components: abstention, validation gates, human fallback with context transfer (Ch 4, 5, 8, 9).
5. Eval & production (~7 min) — the differentiator, volunteered before it’s asked: golden set as a deploy gate (Ch 5), sampled live-traffic judging, business-metric mapping, SLOs and drift watch (Ch 7), A/B for changes. Adaptation questions land here too: tuned voice, retrieved facts (Ch 10).

A worked mock

The senior sentence

The highest-signal moment in the whole interview is one sentence shape:

“I chose X over Y because the requirement is Z — and if Z changed, I’d reconsider.”

“Strict validation over relaxed, because the feared failure is regulatory — if this were an internal tool, I’d flip it.” “pgvector over a dedicated store, because we’re at 200K chunks on existing Postgres — past ~5M, I’d revisit.” That shape demonstrates all four rubric dimensions at once: a correct mechanism, a genuine tension, a requirement-anchored decision, and clean communication. Two cautions from Chapter 10’s discipline: only articulate real tensions (performing balance where one option dominates reads as inexperience), and keep the rubric in view as you speak — Technical Correctness (the mechanisms you name work the way you say), Trade-off Awareness (every choice priced), Evaluation Rigor (Step 5 volunteered, gates named), Communication (the interviewer can follow the map). Score your own mock against those four before anyone else does.

See it: diagnose the gap

Five design-answer excerpts, each with one critical gap. Name the gap and the step or dimension it violates before revealing — this is the grading view of the same framework, and the fastest way to install it.

How this is graded

• Technical Correctness — the components you name behave the way you say: retrieval, grounding, validation, budgets, and the arithmetic when asked.
• Trade-off Awareness — genuine tensions, priced, anchored to requirements, with the reversal condition stated.
• Evaluation Rigor — Step 5 volunteered; golden set as a gate; the difference between offline pass and production safety kept explicit.
• Communication — this chapter’s weighted dimension: the framework narrated, the time budget kept, depth cues followed, and every decision traceable to a requirement the interviewer heard you establish.

Industry variation

• Regulated domains (banking, healthcare, gov) — the feared failure has legal teeth, so validation and escalation are first-class boxes and “design for zero, then optimize the cost” is the expected posture.
• Consumer scale — the deep dive shifts to Chapter 7’s economics: cascades, caches, and SLOs carry the interview; the eval story leans on sampled online judging at volume.
• Frontier-adjacent / platform teams — expect the 100× follow-up (“what breaks at 100× scale?”) and the build-vs-buy fork; the senior answer re-runs the same framework with the new constraint rather than improvising.

Stretch: the skill above the skills

You can design the system. But the interview itself is a system too — a loop with phases, failure modes, and an evaluator whose rubric you now know. And one phase has changed shape recently: the coding rounds where an AI assistant sits next to you, and what’s graded is how you direct and verify it — clarify, decompose, prompt, review, test, explain. What does deliberate practice for that loop look like, and how do you transfer everything this guide built to questions you’ve never seen? That’s Chapter 12 — interview craft and transfer — the last chapter, about the meta-skill.