/Interview

AI interview rounds · Lesson 4 of 8 · 20 min

The LLM fundamentals screen

Study with AI

Part 1Basics

A screen checks whether you've really built with these models. You don't need to derive anything. You do need to know how tokens, context, temperature, embeddings, caching and hallucination affect a product, and to do cost sums without a calculator.

A model is a function that predicts the next token. Almost everything else follows from that. It has no memory between calls, so you send the conversation each time. It has a fixed window, so long input pushes something out. It predicts plausible text, so it can be fluently wrong.

Words you need

Token
The unit a model reads and writes: about 4 characters of English on average.
Context window
The limit on input and output together, per request.
Temperature
How much randomness goes into choosing the next token, on models that expose it. It doesn't make answers true.
Embedding
A list of numbers representing meaning. Similar text gets similar numbers.
Prompt caching
Reusing an unchanged start of a prompt at a lower price. Any change early on breaks the match.
Hallucination
Confident output that's false. It's a consequence of predicting plausible text, so you design around it.

A worked example

The interviewer asks Meera: 'Your assistant gets 1,000 requests a day with 10,000 input tokens each, 8,000 of them cached, and 500 output tokens. Input is $3 per million, cached input is $0.30, and output is $15. What's the monthly bill?' Meera works it through out loud: 2,000 fresh tokens cost 0.6 cents, 8,000 cached cost 0.24 cents, and 500 output cost 0.75 cents. That's about 1.6 cents a request, and about $477 a month. Then she adds that those prices are only the example's, and that she'd look up current ones.

Part 2The question

What do you need to know about models to pass a first technical screen?

By the end of this lesson, you should be able to:

  • Explain tokens, context, embeddings and caching simply
  • Work out the cost of a feature
  • Say what to do when a prompt won't fit

Part 3In detail

Step by step

  1. Learn the vocabulary. The critical basics page has cards for this.
  2. Practise the cost sum. Fresh input, cached input and output have different rates.
  3. Know the limits. Input plus reply must fit the window, so reserve room for the reply.
  4. Know the failure modes. Hallucination, prompt injection, cut-off output, and drift between model versions.
  5. Build one small thing. Nothing prepares you better. The last path builds one.

If a prompt won't fit

In order: send less irrelevant text, retrieve better chunks, summarise old conversation turns, then consider a larger window. Don't silently cut text off: the model won't know what's missing, and neither will you.

Prices and limits go out of date

Don't memorise a specific model's prices or limits. Learn the sum and look up the numbers. Saying 'I'd check the current price sheet' is the right answer.

The code, explained

This is real code, and our tests run it, including the buggy versions, to prove the bugs are real.

Fitting a prompt into the context

Your prompt, the retrieved text and the model's reply all share one limit.

TokenBudget.tsTypeScript
1/** About four characters per token in English. Good for planning; use a real counter before sending. */2export const estimateTokens = (text: string) => Math.ceil(text.length / 4);3 4export interface Chunk {5  id: string;6  text: string;7  score: number;8}9 10/** Take the best chunks that fit, after saving room for the fixed text and the reply. */11export function fitContext(fixedText: string, chunks: Chunk[], contextLimit: number, replyTokens: number) {12  let left = contextLimit - replyTokens - estimateTokens(fixedText);13  if (left <= 0) throw new Error("the fixed text and reply already fill the context");14  const kept: Chunk[] = [];15  const dropped: string[] = [];16  for (const c of [...chunks].sort((a, b) => b.score - a.score)) {17    const need = estimateTokens(c.text);18    if (need <= left) {19      kept.push(c);20      left -= need;21    } else {22      dropped.push(c.id);23    }24  }25  return { kept, dropped, tokensLeft: left };26}
26 linesUTF-8TypeScript
text.length / 4
The usual English rule of thumb. Fine for planning, but use a real token counter before sending.
contextLimit - replyTokens - estimateTokens(fixedText)
Reserve the reply first. Forgetting this is a classic cause of cut-off answers.
dropped.push(c.id)
Records what got left out, so a bad answer can be traced back to it.

Tested by npm run test:modules.

The cost of a request

The sum every LLM screen expects you to do in your head.

TokenBudget.tsTypeScript
1export interface Prices {2  /** dollars per million tokens */3  input: number;4  output: number;5  cachedInput: number;6}7 8export function requestCost(inputTokens: number, cachedTokens: number, outputTokens: number, p: Prices): number {9  const fresh = inputTokens - cachedTokens;10  return (fresh * p.input + cachedTokens * p.cachedInput + outputTokens * p.output) / 1_000_000;11}12 13export const monthlyCost = (costPerRequest: number, requestsPerDay: number) => costPerRequest * requestsPerDay * 30;
13 linesUTF-8TypeScript
inputTokens - cachedTokens
Cached tokens are billed at their own lower rate, so split them out first.
/ 1_000_000
Prices are quoted per million tokens.
costPerRequest * requestsPerDay * 30
Monthly cost is what makes a design decision real.

Tested by npm run test:modules.

What can go wrong

Saying temperature 0 makes it accurate
It reduces randomness. A wrong answer stays wrong.
Forgetting the reply in the context limit
The reply gets cut off, and it looks like a model bug.
Quoting stale numbers
Interviewers can tell. Give the method and say you'd check the numbers.

Part 4Questions and answers

Answer out loud first, then open each one.

Why does a chatbot seem to remember, if models have no memory?
The application sends the conversation again with each request. That's why long chats get more expensive, and why they eventually hit the limit.
What breaks prompt caching?
Any change early in the prompt, such as a timestamp in the system text. Keep fixed content first, and what varies last.
What's an embedding for?
It turns text into numbers so you can find similar meaning by comparing vectors. That's the basis of semantic search.
How do you reduce hallucination in a product?
Give the model the facts in the prompt, ask it to cite them, check the citations in code, let it say 'I don't know', and measure with evals.
What is prompt injection?
Text in the input that tries to act as instructions, such as a web page telling an agent to send data elsewhere. You limit what tools can do, and treat retrieved text as data.

Part 5Going further

Drill: Three sums, out loud (10 min)

  1. A feature uses 3,000 input and 300 output tokens, 50,000 times a day. Work out the monthly cost at $3 and $15 per million.
  2. Now cache 2,000 of the input tokens at $0.30. What's saved?
  3. The window is 8,000 and the reply needs 1,000. Your fixed text is 500. How much is left for retrieved text?

Keep the fundamentals fresh with the critical basics.

Who says so

Checked in November 2026. Vendors and individuals often report what candidates told them, so treat their details as reports.

Next: the design round you're most likely to see. Build a RAG system on a whiteboard.

Next: RAG design →

← The code comprehension round

AI Engineer · AI interview rounds