Blog

Every agent framework, ranked

Every agent framework is a disagreement about how much to trust the model.

Inference engineering

Your GPU is mostly idle during text generation. The entire inference stack exists to fix that.

The annotated microGPT

Karpathy's nanoGPT, annotated line by line.

What happens when you press 'submit' on ChatGPT

From raw internet text to a streaming response: pretraining, alignment, inference.

Attention residuals

The residual connection solved deep learning in 2015. Ten years later, Moonshot AI noticed it also dilutes signal in very deep networks. The fix is 30 lines of PyTorch.

Building PyTorch from scratch

Building PyTorch from scratch in ~1300 lines - reverse-mode autograd, conv2d, optimizers. Trains MNIST. No C++, no CUDA, just closures and NumPy.

My favourite interview questions

Five questions I actually ask ML candidates.

Building RAG for my website

My blogs are too long so I built a RAG over them. Chunking, FAISS, reranking, streaming via Groq - and an eval pipeline because vibes aren't metrics.

The math behind deep learning

The math behind deep learning, with code for every concept.

NumPy + PyTorch from scratch

Ground-up NumPy and PyTorch - we build MNIST classifiers two ways to see how the pieces fit together.

How to read a paper (DeepSeekV3)

Taking DeepSeekV3 apart: MLA, MoE routing, multi-token prediction. And how to read a 53-page paper.

Scaling laws

Maybe intelligence is just a lot of compute applied to a lot of data having a lot of parameters. OpenAI wrote a paper on this.

The YOLO story (v1 → v10)

The full YOLO story from v1 to v10 - grid system, loss function, NMS, mAP, and a PyTorch implementation.

Hello world of neural networks

Neural nets from scratch. NumPy only. Build a digit classifier.

Some Python tricks

Random Python tricks I've picked up.

I forked CPython and added a 'nuke' statement

Forking CPython, modifying the grammar, adding a 'nuke' statement, and other things I found digging through Python's C source.

Things I like