AI interview rounds · Lesson 4 of 8 · 20 min
The LLM fundamentals screen
Part 1Basics
A screen checks whether you've really built with these models. You don't need to derive anything. You do need to know how tokens, context, temperature, embeddings, caching and hallucination affect a product, and to do cost sums without a calculator.
A model is a function that predicts the next token. Almost everything else follows from that. It has no memory between calls, so you send the conversation each time. It has a fixed window, so long input pushes something out. It predicts plausible text, so it can be fluently wrong.
Words you need
- Token
- The unit a model reads and writes: about 4 characters of English on average.
- Context window
- The limit on input and output together, per request.
- Temperature
- How much randomness goes into choosing the next token, on models that expose it. It doesn't make answers true.
- Embedding
- A list of numbers representing meaning. Similar text gets similar numbers.
- Prompt caching
- Reusing an unchanged start of a prompt at a lower price. Any change early on breaks the match.
- Hallucination
- Confident output that's false. It's a consequence of predicting plausible text, so you design around it.
A worked example
The interviewer asks Meera: 'Your assistant gets 1,000 requests a day with 10,000 input tokens each, 8,000 of them cached, and 500 output tokens. Input is $3 per million, cached input is $0.30, and output is $15. What's the monthly bill?' Meera works it through out loud: 2,000 fresh tokens cost 0.6 cents, 8,000 cached cost 0.24 cents, and 500 output cost 0.75 cents. That's about 1.6 cents a request, and about $477 a month. Then she adds that those prices are only the example's, and that she'd look up current ones.
Part 2The question
What do you need to know about models to pass a first technical screen?
By the end of this lesson, you should be able to:
- Explain tokens, context, embeddings and caching simply
- Work out the cost of a feature
- Say what to do when a prompt won't fit
Part 3In detail
Step by step
- Learn the vocabulary. The critical basics page has cards for this.
- Practise the cost sum. Fresh input, cached input and output have different rates.
- Know the limits. Input plus reply must fit the window, so reserve room for the reply.
- Know the failure modes. Hallucination, prompt injection, cut-off output, and drift between model versions.
- Build one small thing. Nothing prepares you better. The last path builds one.
If a prompt won't fit
In order: send less irrelevant text, retrieve better chunks, summarise old conversation turns, then consider a larger window. Don't silently cut text off: the model won't know what's missing, and neither will you.
Prices and limits go out of date
Don't memorise a specific model's prices or limits. Learn the sum and look up the numbers. Saying 'I'd check the current price sheet' is the right answer.
The code, explained
This is real code, and our tests run it, including the buggy versions, to prove the bugs are real.
Fitting a prompt into the context
Your prompt, the retrieved text and the model's reply all share one limit.
1/** About four characters per token in English. Good for planning; use a real counter before sending. */2export const estimateTokens = (text: string) => Math.ceil(text.length / 4);3 4export interface Chunk {5 id: string;6 text: string;7 score: number;8}9 10/** Take the best chunks that fit, after saving room for the fixed text and the reply. */11export function fitContext(fixedText: string, chunks: Chunk[], contextLimit: number, replyTokens: number) {12 let left = contextLimit - replyTokens - estimateTokens(fixedText);13 if (left <= 0) throw new Error("the fixed text and reply already fill the context");14 const kept: Chunk[] = [];15 const dropped: string[] = [];16 for (const c of [...chunks].sort((a, b) => b.score - a.score)) {17 const need = estimateTokens(c.text);18 if (need <= left) {19 kept.push(c);20 left -= need;21 } else {22 dropped.push(c.id);23 }24 }25 return { kept, dropped, tokensLeft: left };26}text.length / 4- The usual English rule of thumb. Fine for planning, but use a real token counter before sending.
contextLimit - replyTokens - estimateTokens(fixedText)- Reserve the reply first. Forgetting this is a classic cause of cut-off answers.
dropped.push(c.id)- Records what got left out, so a bad answer can be traced back to it.
Tested by npm run test:modules.
The cost of a request
The sum every LLM screen expects you to do in your head.
1export interface Prices {2 /** dollars per million tokens */3 input: number;4 output: number;5 cachedInput: number;6}7 8export function requestCost(inputTokens: number, cachedTokens: number, outputTokens: number, p: Prices): number {9 const fresh = inputTokens - cachedTokens;10 return (fresh * p.input + cachedTokens * p.cachedInput + outputTokens * p.output) / 1_000_000;11}12 13export const monthlyCost = (costPerRequest: number, requestsPerDay: number) => costPerRequest * requestsPerDay * 30;inputTokens - cachedTokens- Cached tokens are billed at their own lower rate, so split them out first.
/ 1_000_000- Prices are quoted per million tokens.
costPerRequest * requestsPerDay * 30- Monthly cost is what makes a design decision real.
Tested by npm run test:modules.
What can go wrong
- Saying temperature 0 makes it accurate
- It reduces randomness. A wrong answer stays wrong.
- Forgetting the reply in the context limit
- The reply gets cut off, and it looks like a model bug.
- Quoting stale numbers
- Interviewers can tell. Give the method and say you'd check the numbers.
Part 4Questions and answers
Answer out loud first, then open each one.
Why does a chatbot seem to remember, if models have no memory?
What breaks prompt caching?
What's an embedding for?
How do you reduce hallucination in a product?
What is prompt injection?
Part 5Going further
Drill: Three sums, out loud (10 min)
- A feature uses 3,000 input and 300 output tokens, 50,000 times a day. Work out the monthly cost at $3 and $15 per million.
- Now cache 2,000 of the input tokens at $0.30. What's saved?
- The window is 8,000 and the reply needs 1,000. Your fixed text is 500. How much is left for retrieved text?
Keep the fundamentals fresh with the critical basics.
Who says so
Checked in November 2026. Vendors and individuals often report what candidates told them, so treat their details as reports.
- What applied-AI engineer interviews test Individual, July 2026
Next: the design round you're most likely to see. Build a RAG system on a whiteboard.