The transformer-circuits framework writes a model's output as a sum over paths. In two-layer toy models that sum is finite and enumerable. The paper's own central open question is whether the framework scales — and the obstruction to scaling is a truncation with no tail bound.
Nothing on this page is claimed, certified, enclosed or proved. No kernel, no certificate, no falsifier, no ledger record, and no literature gate has run on it. It names a direction extracted from someone else's published work and what would have to be true for it to be ours. A prospectus that reads like a result is the defect, so this one says so at the top.
Path expansion expresses the output as a sum over end-to-end paths; with attention patterns frozen those paths are linear compositions of weight matrices. At two layers you enumerate them all. At ninety-six you cannot — you keep the paths you can interpret and drop the rest. A truncated expansion with no bound on the discarded remainder is an uncontrolled approximation, and bounding a discarded tail is the one thing this tree does best.
For a given model and a chosen set S of retained paths:
the contribution of every path outside S is bounded in norm by an explicit
ε, computed with outward-rounded arithmetic. An interpretability claim of the form
“this behaviour is implemented by these heads” then carries a number saying how much
of the model's behaviour it does not account for.
A path outside S whose
measured contribution exceeds the bound — found by direct evaluation on a planted example. The
bound must be shown to go red on a deliberately understated ε.
The fetched summary of the paper states that “no eigenvalue/spectral arguments are employed”. My own recollection is that the paper uses OV-circuit eigenvalues to argue copying behaviour in one-layer models. Those cannot both be right about the same text. The summary is a machine reading; the recollection is unsourced. Neither is authority, so nothing on this page rests on the eigenvalue reading, and resolving it requires reading the paper itself.
Written this way deliberately. The most expensive documented error in this tree came from two records disagreeing about one paper and the discrepancy being resolved silently in favour of the convenient one.
Occupancy risk is HIGH and probably fatal. The paper is from 2021 and mechanistic interpretability has moved enormously since — sparse autoencoders, attribution graphs, and successor work from the same group. The question “how much of the model does this circuit explain” is unlikely to be unasked five years on. A literature gate is not optional here, it is the first move, and this direction should be expected to die to it.
Second risk: the source was not read. Everything above rests on a summariser's rendering of the paper, not the paper. Any real work starts by reading it properly.
MLP layers are described as “a major weakness of our work”; layer normalisation is set aside; biases are folded into weights; only attention-only models of at most two layers are analysed. A tail bound over attention paths that ignores MLPs bounds the wrong remainder — which may be the structural objection that kills this before the literature does.
N: unknown, plausibly nonzero as an application of validated numerics to interpretability; A: unknown. Both unverified, and given the two risks above this is the one of the three to gate first and abandon fastest.