Semantic Phonons: Lattice Vibrations in AI Internals I apply the mathematics of phonon modes to study semantic representations in language models. I derive theoretical predictions and verify them in GloVe and Gemma 2B. From Gibbs to GigaBytes: TurboQuant, Rate-Distortion, and Statistical Mechanics A detailed walkthrough of TurboQuant KV compression, along with some thoughts from reading it. Diffusion Geometry for LLM internals I use diffusion geometry to study the residual stream of LLMs. Lattice Gas, Hopfield Networks and Transformer Attention I study a lattice gas in the grand canonical ensemble, a project from my PhD, and show how it connects to transformer models through Hopfield networks. Dendrite Networks A cellular automaton model simulating dendritic growth with parallelized diffusion. Brownian Stocks A fun weekend project where I modeled stock prices using Langevin Dynamics.
