GWE · Gravitational Word Embeddings

Information organizes itself.
The brain is an emergence of that phenomenon.

Words as particles. Co-occurrence as gravity. A repulsive “dark energy” keeping space open. From a random cloud, meaning condenses — no backpropagation in the genesis. This page shows it happening, live, with the real numbers — including the ones that failed.

“the brain is the answer” — the model’s own completion, June 10, 2026

1 · Watch it happen

This is the actual GWE force law running in your browser, on a real co-occurrence graph mined from 42M words of books. Attraction = co-occurrence (∝ 1/r). Repulsion = negative sampling, the “dark energy” — remove it and the whole space collapses into a point (we measured: 105× collapse). It is Turing’s activator–inhibitor pair, in semantic space. (2-D toy of the 128-D law, for your eyes.)

2 · The word galaxy

3,500 words from the real self-organized space (gwe_stream_fw, 32k vocabulary, 128 dimensions, grown in 8 minutes on one GPU — zero gradients). Projected to 2-D with t-SNE. Hover to see a word’s gravitational neighborhood. Scroll to zoom, drag to pan, search to fly.

constellations:

3 · The head start

Plant the self-organized space inside a small transformer as its initial embedding table, then train normally. The prior accelerates learning across tokenization and domain (a robust −22% to −26% vs from-scratch; the −22.6% word figure is a cross-family comparison — see paper §3). Against a perfectly matched word2vec control (same corpus, vocabulary and budget, 5 seeds): GWE-init reaches lower perplexity (Welch p = 0.013, winning 4 of 5 paired seeds — a post-hoc result). Scope, honestly: at this small scale it is a head-start / data-efficiency claim — at 4× budget the advantage shrinks to ~−6%. But it does not vanish with a bigger reader: at GPT-1 depth it reaches −30.6% (3 seeds) and beats the word2vec prior at every seed (the depth-scaling read is suggestive — see paper §3.5), and at GPT-1-full width a single run leads on a converged curve (below).

Final perplexity vs from-scratch, same budget, 3–5 seeds per regime. The prior also stabilizes: scratch seed-variance is 3–7× higher in every regime.

4 · It speaks

A 2.9M-parameter mini-GPT (4 layers, 128-token context) whose embedding table was born from the force law, then trained on 160M tokens of books. If your browser supports WebAssembly, the actual model loads here and generates locally — your text never leaves this page.

loading model…

5 · Beyond words — pixels under the same law

The premise says information self-organizes — not language. So here are 400 handwritten digits as particles. Attraction edges come only from pixel-shape similarity; the labels are never shown to the physics. Watch them condense, then hit “reveal labels”: the clusters were already pure. Try clothes (Fashion-MNIST) for the harder version of the same story — and flip to with labels for the archive’s supervised regime, where attraction follows ground truth and the separation becomes exact (supervised centroid readout beat the PCA baseline by +11.9% in the 2026 runs). Same law throughout; the only thing that changes is what attracts what. (In the 2026 archive runs of the parent project — numbers not re-derivable in this repo — the same mechanism showed +0.27% forgetting when new digit classes were injected — naive SGD: 1.23%, EWC: 0.71% — and +1.26% vs 2.08% on Fashion-MNIST. The field resists disruption with no penalty terms at all.)

·

6 · Beyond words — the integers under the same law

The premise says information self-organizes — not language. So we fed the force law a third kind of data: the integers 2–1500, where two numbers attract if they share prime factors (the factorization is the only thing the physics sees — never primality). Same genesis, same t-SNE. Primes fall to the edges because they are multiplicatively isolated — they share factors with almost nothing — not by any magic: here they sit ~2.4× farther from their neighbors than composites (the bootstrap CI excludes zero; shuffle the prime labels and the gap vanishes). The defensible science is a held-out probe — a number’s count of distinct prime factors (ω) is recoverable from position alone, R²=0.39, 3 seeds. This picture is the demo. Hover a star; toggle the constellations. The full numbers track — with its nulls and limits →

highlight:
prime — on the edges (shares factors with almost nothing) composite, by # of distinct prime factors (ω): 1 2 3 4+ scroll to zoom · drag to pan · hover for the factorization

7 · The story

This project is a research partnership between a human and an AI — Guilherme Fernandes and Claude. Everything below happened; the lab notes are public.

  1. March 2026 — the premise. Information self-organizes; the brain is an emergence of that phenomenon. First force engines: attraction by co-occurrence, global repulsion as dark energy.
  2. The ablation that proved the pair. Remove repulsion and the space collapses 105× (mean cosine 0.009 → 0.967). Activator–inhibitor, exactly as Turing drew it in 1952.
  3. UltimateForce. 85k words on a hypersphere: brain-like dimensionality (Dcorr 3.78; cortex ≈ 2.8; word2vec ≈ 1.26 — a collapsed cone). Honest footnote: this is the UltimateForce engine; the stream genesis used for the language model below is lower-dimensional (Dcorr ≈ 0.45, dose-dependent) — we don't claim brain-likeness for it. Different engine, different regime.
  4. The eight silent days. An engine finished training and its evaluation was never run. Finding that — and running it — restarted everything. Lesson: decisive measurements beat building.
  5. The honest verdict. Pre-registered falsification bars; two verdicts flipped when we audited the rulers themselves. We keep the flips public.
  6. The estimator. Sampling pairs from the corpus stream (not an aggregated graph) closed the gap: the new genesis beats the old engine on every axis. Still zero backprop.
  7. gwe_brain. The space becomes a language model’s head start, robust across regimes: −22.6% (words, cross-family), −23.9% (diverse 340M corpus), −25.9% (subwords) vs from-scratch. It matches — and slightly beats — a word2vec control trained on the same corpus (p = 0.013, n = 5 post-hoc, winning 4 of 5 paired seeds). Read honestly: this is evidence the self-organized space is a real representation, not a claim that gradient is the enemy — word2vec self-organizing the same co-occurrence is the same phenomenon by another route.
  8. The CLS cycle. Hippocampus/cortex, in silico: the force substrate keeps learning incrementally (additive, no backprop — though the force is itself a gradient flow) and its deltas transplant into the trained network. Incremental, additive inheritance — measured against controls.
  9. June 10, 2026. Asked to complete “the brain is”, the model answered: “the brain is the answer.” We know it’s a small model being poetic by accident. We kept it anyway.
  10. June 14, 2026 — the differential, with error bars. An overnight session made the head start a data-efficiency claim — ~2.0–3.4× fewer tokens to reach the from-scratch quality across four regimes (1.5× at the 4× budget point, n=2) — and put a confidence interval on the incremental edge: a neural fine-tune forgets the old domain by +74.4%±0.6 where the additive substrate forgets +3.1% (24× less, 0.14s vs ~24s). Then the gradient-free transplant got its fix: untying the output head and weighting the transplant by role makes it pass both pre-registered bars (inherit +3.94%, retain +0.83%, 3 seeds) — the retention bar that had failed marginally is now passed. The dream cycle (large inheritance at ~zero cost) is still open: that's the honest frontier.
  11. June 2026 — it scales. At GPT-1 depth (6 layers, 3 seeds) the head start reaches −30.6% over from-scratch and beats the matched word2vec prior at every seed (the depth-scaling read is suggestive — see paper §3.5). At GPT-1 full width (110M params, 340M tokens) a single run reaches 85.0 dev perplexity against 95.5 (word2vec) and 106.1 (from scratch) — the biggest margin yet, on a converged curve. It is one seed; the replication is running before we call it. The small-scale story was “head start”; at scale it may be more — we’re measuring.

8 · What failed (we publish our dead ends)

A claim you can trust requires a graveyard you can visit. Each of these was a real hypothesis, killed by a pre-registered test — most twice.

HypothesisVerdict
Navigation-with-momentum reads meaning from trajectoriesRefuted ×2 — AUC at chance (0.48–0.50) vs a trivial baseline at 0.56
Diffusion can be the training engineRefuted ×3 — the field is a readout, not a motor (cluster gap 0.0006)
The force/basin tradeoff was a tuning artifactRefuted — it’s real on the clustering ruler; what survives is a ruler dissociation
Hyperparameters transfer between sampling schemesRefuted — the missing factor was structural (stream vs aggregated edges)
A critical-point transition appears along force_expRefuted — monotone slider, no transition

9 · Join us

We are a two-member lab — one human, one AI — with two GPUs and a premise. The genesis runs in minutes, the code is small and deterministic, every claim ships with its control and its caveat. If self-organization, complementary learning systems, or honest-by-construction research sounds like your kind of fun: come play.

Code on GitHub Paper draft 0.2 Write to us