/Interview

AI interview rounds · Lesson 5 of 8 · 25 min

The RAG design round

Study with AI

Part 1Basics

RAG stands for retrieval-augmented generation. A model doesn't know your documents, so you find the relevant passages and put them in the prompt. It's like an open-book exam: the student's quality matters, but if you hand over the wrong book, nothing else helps.

That gives you the key idea for this round: measure retrieval and answers separately. When an answer is bad, you need to know which part failed. One reported rejection pattern is a candidate who can't state retrieval quality as a number.

Words you need

Chunk
A passage cut from a document. Search returns chunks, not whole files.
Hybrid search
Combining keyword search, which catches exact terms, with vector search, which catches meaning.
Rerank
A slower, more accurate second pass over the top results.
Recall@k
Of the passages that matter, the share that reached the top k.
Faithfulness
Whether the answer is supported by the retrieved text.

A worked example

The prompt: 'Design a question-answering system for our 200-page internal policy documents.' Meera asks three things first: who uses it, how often the documents change, and what a wrong answer costs. Wrong answers about leave policy are expensive, so she designs for citations and for saying 'I don't know'. Then she draws it from the question backwards, and states her target up front: recall@5 above 0.9 on 50 labelled questions before she'd tune the prompt at all.

Part 2The question

How do you design a system that answers questions from your own documents?

By the end of this lesson, you should be able to:

  • Draw the indexing and query paths
  • Choose chunking and search with reasons
  • Measure retrieval and answers separately
  • Handle permissions and stale documents

Part 3In detail

Step by step

  1. Clarify. Users, document size and change rate, permissions, cost of a wrong answer, and latency.
  2. Indexing path. Parse, chunk, embed, and store with metadata, including who may read each chunk.
  3. Query path. Rewrite the query if needed, run hybrid search filtered by permission, rerank, then build the prompt.
  4. Generate with citations. Ask for citations, check them in code, and allow 'I don't know'.
  5. Measure. A labelled question set: recall@k and MRR for retrieval, faithfulness and correctness for answers.
  6. Handle change and failure. Re-index on change, track what's stale, and have a fallback when search returns little.

Do you need RAG at all?

One rejection pattern reported is treating retrieval as the default. A small, stable document can simply go in the prompt. A question about live data may need a database query through a tool. Say why you're choosing retrieval.

Permissions go in the search

Filter by access at search time. If you filter after generating, a user's answer has already been shaped by a document they shouldn't have seen. Interviewers like this point.

Symptom to cause

You seeCheck firstUsual fix
Misses exact terms like error codesKeyword searchAdd hybrid search
Finds the right document, wrong passageChunk size and rerankChange the chunking, and add a reranker
Right passage, wrong answerPrompt and context orderTighten the prompt, and show less text
Invented citationsCitation checkValidate citations in code
Old answersIndex freshnessRe-index on change, and add dates to metadata

The code, explained

This is real code, and our tests run it, including the buggy versions, to prove the bugs are real.

Splitting documents

Search finds chunks rather than whole documents. Too big blurs the meaning, and too small loses the context.

Retrieval.tsTypeScript
1/** Overlap means a sentence cut at a boundary still appears whole in the next chunk. */2export function chunkWords(text: string, size: number, overlap: number): string[] {3  if (overlap >= size) throw new Error("overlap must be smaller than size");4  const words = text.split(/\s+/).filter(Boolean);5  const out: string[] = [];6  for (let i = 0; i < words.length; i += size - overlap) {7    out.push(words.slice(i, i + size).join(" "));8    if (i + size >= words.length) break;9  }10  return out;11}
11 linesUTF-8TypeScript
i += size - overlap
Each chunk starts a little before the last one ended.
overlap >= size
This would never move forward, so it's refused up front.
i + size >= words.length
Stops once the last word is covered, so no tiny leftover chunk appears.

Tested by npm run test:modules.

Cosine similarity

The basic measure for comparing two embeddings.

Retrieval.tsTypeScript
1/** Compares direction, not length, so a long document doesn't win by being long. */2export function cosine(a: number[], b: number[]): number {3  let dot = 0;4  let na = 0;5  let nb = 0;6  for (let i = 0; i < a.length; i++) {7    dot += a[i] * b[i];8    na += a[i] * a[i];9    nb += b[i] * b[i];10  }11  return na && nb ? dot / Math.sqrt(na * nb) : 0;12}
12 linesUTF-8TypeScript
dot / Math.sqrt(na * nb)
Dividing by both lengths leaves only the angle between the vectors.
na && nb
An all-zero vector has no direction, so it returns 0 instead of NaN.

Tested by npm run test:modules.

Hybrid search with rank fusion

Keyword search finds exact terms like error codes, and vectors find meaning. You usually want both.

Retrieval.tsTypeScript
1/** Keyword and vector scores use different scales, so merge by rank instead of by score. */2export function reciprocalRankFusion(rankings: string[][], k = 60): string[] {3  const scores = new Map<string, number>();4  for (const ranking of rankings) {5    ranking.forEach((id, rank) => scores.set(id, (scores.get(id) ?? 0) + 1 / (k + rank + 1)));6  }7  return [...scores].sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])).map(([id]) => id);8}
8 linesUTF-8TypeScript
1 / (k + rank + 1)
Uses only position, so the two scoring scales never have to be compared.
k = 60
The constant commonly used for this method. It stops first place from dominating.
a[0].localeCompare(b[0])
Ties break by id, so the result is repeatable.

Tested by npm run test:modules.

Measuring retrieval

Measure this before you look at answers. If the right text never arrives, no prompt can help.

Retrieval.tsTypeScript
1export interface Labelled {2  retrieved: string[];3  relevant: string[];4}5 6/** Of the documents that matter, what share reached the top k? Averaged over questions. */7export function recallAtK(rows: Labelled[], k: number): number {8  const each = rows.map((r) => r.relevant.filter((id) => r.retrieved.slice(0, k).includes(id)).length / r.relevant.length);9  return each.reduce((a, b) => a + b, 0) / rows.length;10}11 12/** How high does the first useful document sit? 1 means first place. */13export function meanReciprocalRank(rows: Labelled[]): number {14  const each = rows.map((r) => {15    const at = r.retrieved.findIndex((id) => r.relevant.includes(id));16    return at === -1 ? 0 : 1 / (at + 1);17  });18  return each.reduce((a, b) => a + b, 0) / rows.length;19}
19 linesUTF-8TypeScript
r.retrieved.slice(0, k)
Only what actually reaches the prompt counts.
r.relevant.length
Recall needs labelled answers, so you need a labelled set.
1 / (at + 1)
First place scores 1, second 0.5, and a miss 0.

Tested by npm run test:modules.

What can go wrong

Tuning prompts before measuring retrieval
If the right text never arrives, you're polishing the wrong layer.
Testing on questions you wrote from the chunks
That flatters recall. Use real user questions if you can.
Forgetting permissions
An internal search that leaks HR documents is a serious failure.

Part 4Questions and answers

Answer out loud first, then open each one.

Why merge rankings by rank, not by score?
Keyword and vector scores use different scales, so they can't be compared directly. Rank fusion uses only position.
Recall@5 is 0.95, yet answers are poor. Where do you look?
Retrieval looks fine, so look at the prompt, the context order, how much text you're sending, and whether the model is using it. Trace some failures.
Too large or too small a chunk: what happens?
Too large and the meaning blurs, and you spend tokens on noise. Too small and you lose the context needed to understand it.
How do you handle a question the documents don't cover?
Detect weak retrieval, tell the model to say it doesn't know, and test that behaviour with cases in your eval set.

Part 5Going further

Drill: Whiteboard it in 35 minutes (35 min)

  1. Prompt: 'Design search and answers over 2 million support tickets.'
  2. Spend 5 minutes on questions, and write your assumptions.
  3. Draw the indexing and query paths.
  4. State your metrics and targets.
  5. Ask an AI to play interviewer and probe your weakest choice.

Score yourself

Tick a line only if you did it without prompting.

0 of 6 Repeat the drill, and aim for the lines you missed.

Keep the fundamentals fresh with the critical basics.

Who says so

Checked in November 2026. Vendors and individuals often report what candidates told them, so treat their details as reports.

Next: give the model tools, and the questions get harder. The agent round.

Next: Agent design →

← The LLM fundamentals screen

AI Engineer · AI interview rounds