AI systems · 48 modules
AI systems, part by part.
Each module opens with the whole idea, states the question, walks the mechanism, then tests you. Use Explore to filter by mode, tier or time.
Mobile systems
Startup, observability, security, location, push, experimentsTier 2 · Mechanisms · 15 minApp Startup: Cold Start From Tap to First FrameStartup is the first performance number users feel, and most of it is work you chose to do too early.Tier 2 · Mechanisms · 15 minMobile Observability: Crashes, ANRs, Traces and Rollout GatesYou cannot roll back a phone, so telemetry has to catch a bad release while only 1% of users have it.Tier 2 · Mechanisms · 15 minMobile Security: Tokens, Keystore, Pinning and TamperingAssume the phone can be lost, rooted or reverse-engineered, then decide what an attacker still cannot do.Tier 3 · Edge cases · 45 minLive Location Tracking for a Driver AppA location stream combines every mobile constraint at once: battery, background limits, bad networks and scale.Tier 2 · Mechanisms · 45 minPush Notification Platform at ScalePush looks like one API call until you need priority, retries, per-user limits and delivery you can measure.Tier 2 · Mechanisms · 40 minFeature Flag and Experimentation SDKFlags are how mobile teams change behaviour without a store release, and bad assignment logic silently ruins experiments.Tier 2 · Mechanisms · 15 minReact Native New Architecture: JSI, Fabric, TurboModules and ThreadsMost older React Native advice is about the bridge. Knowing what replaced it shows which advice still holds.Tier 2 · Mechanisms · 15 minReact Native State: Server Cache, Client Store and Render StormsPutting every kind of state in one global store makes one change redraw half the screen.Tier 2 · Mechanisms · 15 minModular React Native for Many Apps: The Modularity LadderCodebases rarely fail from network speed. They fail from merge conflicts and features that can't be removed.Tier 3 · Edge cases · 15 minMemory Leaks in React Native: Three Heaps, One StaircaseA leak can hide in the JavaScript heap, the native heap or the boundary between them, and each needs a different tool.Tier 3 · Edge cases · 20 minOffline Sync in React Native: What to Persist and WhereA queue held in memory loses unsent changes when the OS kills the app. Persist the mutations, and keep them atomic with the local write.Tier 3 · Edge cases · 15 minThe 95/5 Rule and Performance Budgets in CIMost code can stay in React Native. A few pieces belong in native code, and CI should stop performance from slipping.
Model serving and inference
PagedAttention, KV cache, vLLM, continuous batchingTier 1 · Whole picture · 35 minvLLM High-Throughput Engine ArchitecturePagedAttention treats the KV cache like virtual memory, so one GPU holds 2–4× more concurrent requests.Tier 2 · Mechanisms · 15 minContinuous Batching & Iteration-Level SchedulingIteration-level scheduling ends the wait for a batch's longest request; prefill stalls become the next bottleneck.Tier 2 · Mechanisms · 15 minSpeculative Decoding Target-Draft Pair VerifierModified rejection sampling accepts draft tokens without changing the target model's distribution.Tier 1 · Whole picture · 10 minvLLM Automatic Prefix Caching ArchitectureChained block hashes skip prefill for shared system prompts, and salts keep tenants apart.
Distributed training
Data and tensor parallelism, DeepSpeed ZeRO, FSDPTier 2 · Mechanisms · 12 minTensor Parallelism & Ring-AllReduce CommunicationEvery transformer layer pays for two collectives on NVLink, which explains why tensor parallelism stays inside one server.Tier 3 · Edge cases · 35 minDeepSpeed ZeRO-3 Memory PartitioningShards optimiser state, gradients and weights across GPUs, trading extra communication for models that otherwise cannot fit.Tier 1 · Whole picture · 35 minPyTorch Fully Sharded Data Parallel (FSDP)PyTorch's native sharding: wrap per block, and each block gathers its weights only while it runs.
Fine-tuning (PEFT)
LoRA, QLoRA, adapter merging, DPO, UnslothTier 1 · Whole picture · 30 minLow-Rank Adaptation (LoRA) Weight DecompositionTrains two thin matrices instead of every weight, so fine-tuning fits one GPU and merges with no added inference latency.Tier 2 · Mechanisms · 35 minPyTorch LoRA Linear Layer ImplementationWrites a drop-in replacement for nn.Linear with trainable low-rank adapters and dropout.Tier 3 · Edge cases · 15 minQLoRA NormalFloat4 & Double QuantizationNF4 and double quantisation shrink a 7B base to about 3.4 GiB, so training fits a small GPU.Tier 2 · Mechanisms · 15 minDirect Preference Optimization (DPO) Loss FunctionTrains on preference pairs directly, with no separate reward model or PPO loop.Tier 2 · Mechanisms · 10 minMerging LoRA Adapters into Base Model WeightsBakes an adapter into the weights for faster serving, and shows when merging is the wrong call.Tier 3 · Edge cases · 10 minUnsloth Gradient Calculation AccelerationUnsloth claims faster, lighter fine-tuning through patched model code. Verify its numbers on your own hardware.
Vector search and RAG
HNSW, hybrid search, chunking, GraphRAGTier 1 · Whole picture · 30 minHierarchical Navigable Small World (HNSW) GraphsLayered small-world graphs answer nearest-neighbour queries in milliseconds, trading exact recall and memory for speed.Tier 1 · Whole picture · 10 minReciprocal Rank Fusion (RRF) Hybrid SearchMerges BM25 and vector rankings without normalising scores that live on different scales.Tier 2 · Mechanisms · 14 minParent-Child Hierarchical Context RetrievalEmbed small chunks for precise matching, and send large parents to the model for enough context.Tier 3 · Edge cases · 40 minEntity-Relation Graph Mining & GraphRAG PipelineEntity graphs and community summaries answer corpus-wide questions that chunk retrieval cannot.Tier 1 · Whole picture · 35 minRedis Vector Semantic Response CachingAnswers repeat questions from a vector cache, cutting LLM cost, as long as false hits stay rare.Tier 2 · Mechanisms · 10 minInverted Indexing & BM25 Scoring EngineInverted indexes and BM25 still win on exact terms: product codes, names and error strings.Tier 3 · Edge cases · 10 minBGE-Reranker Multi-Stage Retrieval PipelineRetrieve 100 with fast search, rerank to 5 with a cross-encoder, and degrade safely on timeout.Tier 1 · Whole picture · 10 minSelective Context Window Token CompressionSpend the token budget on evidence: prune sentences and summarise history before they reach the model.
GPU kernels and low-level systems
Triton, FlashAttention, memory trafficTier 3 · Edge cases · 45 minBlock-Allocated PagedAttention Triton KernelBuild the block table and the kernel that reads it, so multi-tenant batches never fragment GPU memory.Tier 2 · Mechanisms · 35 minFused Flash-Softmax Triton KernelFusing softmax into one kernel cuts memory reads from 4n to n and writes from 2n to n.Tier 1 · Whole picture · 10 minTriton Compiler JIT Codegen PipelineFollows one Python function down to GPU binary, which explains caching, specialisation and autotuning.
Evaluation and benchmarks
RAGAS, LLM-as-a-judge, eval gatesTier 1 · Whole picture · 25 minLLM-as-a-Judge Evaluation & Calibration ArchitectureLLM judges scale evaluation, but only once you correct position, verbosity and self-enhancement bias.Tier 1 · Whole picture · 12 minFaithfulness, Answer Relevance & Context Recall MetricsSeparates retrieval failures from generation failures, so you fix the right half of a RAG system.
LLM safety and guardrails
Prompt injection, Llama Guard, schema validationTier 2 · Mechanisms · 30 minLlama-Guard Jailbreak & Safety ClassifierModerates input and output with a dedicated classifier, and fails closed whenever actions are at stake.Tier 2 · Mechanisms · 35 minGuardrails AI Structured Pydantic Output ValidationNever passes unvalidated model output downstream: validate against a schema and re-ask with the exact errors.Tier 1 · Whole picture · 35 minAutomated Adversarial Red-Teaming SuitesTurns one-off safety testing into a regression suite that gates every release.
Multimodal systems
ViT, CLIP, visual reasoning benchmarksTier 3 · Edge cases · 35 minEvaluating Multimodal LLMs on Visual ReasoningMany vision benchmark items can be answered without the image. Control for that before trusting a score.Tier 1 · Whole picture · 12 minCLIP and ViT Multimodal EmbeddingsCLIP places images and text in one vector space, which enables image search with no labels.
Prompt engineering
Chain of thought, few-shot, structured outputMLOps and infrastructure
SLURM, Ray Serve, NCCL, checkpointingTier 2 · Mechanisms · 15 minSLURM GPU Cluster Job Scheduling & Failure RecoveryRequeues multi-node training on failure, and resumes from the last checkpoint with no one watching.Tier 3 · Edge cases · 40 minAsync Zero-Redundancy Optimizer CheckpointingSaves sharded state in the background, so checkpoints stop stalling GPUs.Tier 2 · Mechanisms · 10 minNCCL Inter-Node Transport Protocols & InfinibandReads an NCCL debug log and finds the moment training silently falls back to TCP.Tier 3 · Edge cases · 35 minRay Serve Distributed Actor DeploymentRuns LLM replicas as autoscaled actors, with health checks and per-replica request limits.