Research & experiments

Building things, and writing down what actually happened.

Software, automations, and whatever experiment I'm running this week.

01
· 10 min read

Grokking on real astronomical data

Grokking has mostly been shown on clean, synthetic tasks. I trained a tiny transformer on real planetary orbits, diagnosed two dataset confounds that produced a fake negative result, got a genuine 24,000-step delayed generalization once they were fixed, then ran 100 seeds to check how often that actually happens.

02
· 6 min read

HSI: Hierarchical Sparse Intelligence

A proof-of-concept multi-agent pipeline where 22 specialist classifiers pre-process every query before it reaches an LLM, and what happened when I actually built and benchmarked it.

03
· 8 min read

Testing a slime-mould idea against Fisher information

I bet a biology-inspired importance signal could beat the standard method at stopping a neural network from forgetting. On the easy benchmark it did, then it failed on the hard one. Here is the honest arc, and the mechanism that explains both.