Towards the Modern Transformer Architecture
Tracing how the 2017 Transformer evolved into the recipes used by modern frontier models — norms, activations, attention variants, RoPE, KV cache, and more.
Jul 22, 2026 · 36 min
Notes on systems, protocols, and models. Full posts, rendered here from the original markdown.
Tracing how the 2017 Transformer evolved into the recipes used by modern frontier models — norms, activations, attention variants, RoPE, KV cache, and more.
Jul 22, 2026 · 36 min
The WebSocket protocol, handshake to frames, and how bidirectional sockets actually work.
Aug 25, 2025 · 31 min
Building interactive UIs from scratch in C++ with FernKit’s renderer and widgets.
Aug 3, 2025 · 10 min
Shamir’s secret sharing and multi-party computation for splitting a private key.
Jul 5, 2025 · 23 min
How TrueType fonts are parsed and pixels get onto the screen.
Jun 14, 2025 · 36 min
How AES encrypts a block, from S-boxes through the last round.
Apr 1, 2025 · 26 min
Building Rosenblatt’s perceptron in Flutter, with the geometry drawn out.
Feb 27, 2025 · 26 min
Clean architecture and BLoC in Flutter, past the introductory examples.
Jan 5, 2025 · 25 min
The BLoC pattern in Flutter, with small examples you can run.
Dec 13, 2024 · 25 min
How the Linux scheduler allocates CPU, and why probabilistic methods show up.
Nov 1, 2024 · 37 min
How Android Runtime compiles, garbage-collects, and changes what apps can assume.
Sep 3, 2024 · 6 min
Installing, configuring, and administering Arch Linux from a blank disk.
Aug 16, 2024 · 28 min