/Practice

AI systems quick hits reels

AI systems quick hits1 / 4

Filter before or after the search?

Post-filter
  • Search first, filter results
  • Fast and simple
  • May return fewer than k
  • Recall drops on rare values
Pre-filter
  • Filter first, search the rest
  • Always k valid results
  • Slow if done by brute force
  • Breaks graphs at low selectivity
Filter-aware indexes keep links for filtered traversal.
Pre-Filtering vs Post-Filtering in Vector Stores
Strict filters can disconnect an HNSW graph.
#vector-db#filters
AI systems quick hits2 / 4

Debug a CUDA out-of-memory error

1

Read the numbers

torch.cuda.memory_summary() splits allocated from reserved. A large gap points to fragmentation.

2

Record a snapshot

torch.cuda.memory._record_memory_history(), then _dump_snapshot(). View it at pytorch.org/memory_viz.

3

Find the holder

Keeping loss instead of loss.item() retains the whole graph.

4

Reduce fragmentation

Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.

PyTorch CUDA Memory Allocation & OOM Diagnostics
Allocated: live tensors. Reserved: what the allocator holds.
#pytorch#memory
AI systems quick hits3 / 4

Spot a context hallucination

1

Split into claims

Break the answer into single statements.

2

Check each claim

Does the retrieved context support it?

3

Score faithfulness

Supported claims divided by all claims.

4

Find the cause

Evidence present but ignored: fix generation. Evidence missing: fix retrieval.

Spotting Context Hallucinations in RAG
A confident answer can still have no support.
#rag#evals
Real-Time Factuality Verification Shields