AI systems quick hits reels
Filter before or after the search?
Post-filter
- Search first, filter results
- Fast and simple
- May return fewer than k
- Recall drops on rare values
Pre-filter
- Filter first, search the rest
- Always k valid results
- Slow if done by brute force
- Breaks graphs at low selectivity
Filter-aware indexes keep links for filtered traversal.
Pre-Filtering vs Post-Filtering in Vector Stores
Strict filters can disconnect an HNSW graph.
Debug a CUDA out-of-memory error
1
Read the numbers
torch.cuda.memory_summary() splits allocated from reserved. A large gap points to fragmentation.
2
Record a snapshot
torch.cuda.memory._record_memory_history(), then _dump_snapshot(). View it at pytorch.org/memory_viz.
3
Find the holder
Keeping loss instead of loss.item() retains the whole graph.
4
Reduce fragmentation
Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.
PyTorch CUDA Memory Allocation & OOM Diagnostics
Allocated: live tensors. Reserved: what the allocator holds.
Spot a context hallucination
1
Split into claims
Break the answer into single statements.
2
Check each claim
Does the retrieved context support it?
3
Score faithfulness
Supported claims divided by all claims.
4
Find the cause
Evidence present but ignored: fix generation. Evidence missing: fix retrieval.
Spotting Context Hallucinations in RAG
A confident answer can still have no support.