Writing

Explorations of systems, protocols, and models, with explanations built from first principles.

Towards the Modern Transformer Architecture

Tracing how the 2017 Transformer evolved into the recipes used by modern frontier models — norms, activations, attention variants, RoPE, KV cache, and more.

Jul 22, 2026 · 36 min read

Hashnode