Independent AI research
A compact, repetition-resistant language model built from a self-similar tree over channels.
Instead of stacking layers, FractalPI processes language at many channel scales inside one finite tree.
Each token keeps its full representation. Its channels are divided into progressively smaller groups, processed at six scales, then merged back through the transpose of the learned route.
1 → 2D channel groups
02ContextRouter assigns every token
03Down/up weights are shared
The experiments support a narrower, more useful claim: FractalPI leads its size class and resists repetition at conservative learning rates. The larger dense model still achieves the best single-pass loss.
Lower BPB is better. FractalPI does not reach parity with the 1.7× larger dense model; it outperforms the paper’s parameter- and FLOP-matched structural controls.
Matched total capacity: 133M fractal vs 127M dense, one run per model, conservative LR. This is resistance to generalization collapse—not immunity to memorization.
Both models fit at this width.
The model estimates follow the paper’s embedding + quadratic block scaling. At d=8192, the reported 5.2B dense optimizer state is ≈78 GB; the 2.0B fractal still trains at batch 1.
Routing is per token and context-conditioned. Replacing it with a static permutation costs +0.37 BPB—the largest ablation in the paper.
The current study reaches 133M parameters. These are promising measurements, not a theorem of fractal intelligence.
FractalPI grows from years of research at the intersection of nonlinear dynamics, complex systems and intelligent computation.
FULL GOOGLE SCHOLAR PROFILE ↗Krzysztof Urbanowicz, Andrzej Urbanowicz & Mateusz Mach
Albert Śledzianowski, Krzysztof Urbanowicz, Wojciech Glac, Renata Slota, Maria Wojtowicz, Monika Nowak & Andrzej Przybyszewski
Albert Śledzianowski, Jerzy P. Nowacki, Andrzej W. Przybyszewski & Krzysztof Urbanowicz
Explore a new geometry for machine intelligence.