Bounded state
Fixed size. It does not grow with the length of the history.
Axiogram Labs · pre-publication
Models that keep what they learn.
Two lines of work. Machine learning systems that carry a bounded internal state and keep rewriting it while they run, and exact causal measurement of what models are actually computing, on forty-six public ones.
Boundedfixed size, whether it has been running a minute or a month
Writablethe model writes into it while it runs
Persistentwhat it wrote survives the session that produced it
Measuredwhat it computes is checked by exact ablation
Always running
It stays up between requests and carries its state forward.
Yours to run
Predictable state cost is what makes your own hardware realistic.
Checked by ablation
Exact intervention, against controls that are allowed to fail.
Two lines · one question
The research
Can a system carry its own experience as part of its computation, and can anyone verify what that computation is doing?
01
Stateful
machine learning
A model with a state of its own that stays writable while it runs. The hard problem is selection: what becomes durable, what gets consolidated, what can safely be forgotten, and what happens when a new write contradicts an old one.
- No history-sized context window
- Runtime learning lands in the state, weights stay frozen
- Continuous operation is the target
02
Mechanistic
interpretability
Exact causal accounts of what public models compute, taken at mechanism level, below the behavior. The panel spans transformers and the recurrent and state-space architectures the first line is about, which is why it was built that way.
- Forty-six models across fourteen families
- Exact ablation where the proxies disagree
- Findings kept next to the ones that died
46
public models
in the interpretability panel
14
model families
transformer, recurrent and state-space
0.16B to 14B
parameter range
pythia-160m up to Qwen2.5-14B
458,752
neurons per token
exact-ablated in the generation census
Architecture
Where the state lives
The target is a bounded state that takes part in the computation and stays writable while the model runs. The diagram puts that beside the request-and-response arrangement it replaces.
The through-line
Where the margin is thin, the nearest thing decides
A reasoning failure traced token by token committed to a wrong number on a 2.5 percent differential. On a published set of moral dilemmas, swapping which option is printed first flips roughly 42 percent of contested choices while carrying about 0.015 bits. Two different subjects, one structure: fragility tracks the margin.
Choices come apart near indifference.
Exchange the two options and a model should pick the same action under the other letter. On clear-cut dilemmas it does. On genuinely contested ones it barely clears chance.
Most of the write misses the decision.
When a model commits to an answer, almost everything its components write into the residual stream is held orthogonal to the direction that decides it, so magnitude-weighted attribution ranks inert scaffold.
Charts plot reported endpoints only. Where a value is approximate it is marked approximate, and no intermediate points are inferred.
Why this work
A model worth owning.
A stateless model you run locally is the same model on your last day with it as on your first. One that accumulates on your hardware, from your data, becomes specific to your problems without any of that history leaving the machine.
Bounded state is what makes the cost of running one predictable, and predictable cost is what makes your own hardware a realistic place to put it.