Pre-design idea note — sits before a design document, not in place of one. Records a direction for making the notes system's retrieval-by-navigation thesis hold up at larger scale. Speculative; not a commitment.
The system's central retrieval claim (see structured-context-protocol) is that you reach the right note by navigation — mnemonic keys you can carry in your head, plus JSONHTL links embedded in the documents — rather than by vector similarity. The HT in JSONHTL is load-bearing. This is more precise and more debuggable than RAG, but it has an obvious scaling objection: curated hubs bloat and the curation burden grows. CONTENTS is already long and climbing. This note proposes two answers, both compile-time-built and query-time-traversed, that keep similarity search confined to extraction and unknown-frontier discovery.
A book carries two indices for a reason: a table of contents (where a thing lives) and a back-of-book index (what a thing is made of). The same idea surfaces in chapters that live far apart in the ToC, and only the index catches that. The notes system currently has only the first, and it is doing all the work.
CONTENTS is long because it is flat: one hub carrying every leaf. The fix is book structure — a thin top index pointing to section/part indices, those to chapter indices, notes as the leaves. No single hub carries more than its level, so none bloats, and every note stays one short hop from its parent — which is the exact condition that makes navigation beat search. This is the same tree discipline already documented in the workflow (plan the parent/child split first; avoid the megadoc trap), applied to the index itself.
The novel layer. Treat a note as a composite and the concepts it is about as its factors. The concept index is the inverse map: given a concept, list every note that has it as a factor — a back-of-book index built by decomposition at ingest, not by hand. TRIGGERS is a hand-built prototype of the same shape (situation → note, classification by curation); this generalises it and automates the extraction.
The concept labels are not keywords mapped to discrete primes. They are directions in embedding space, extracted by greedy sparse decomposition — this is matching pursuit (Mallat & Zhang, 1993): express a vector as a sparse sum of atoms drawn from a redundant dictionary, peeling the residual at each step. The loop, per note:
1. Embed the note. 2. Find the 2–3 dictionary atoms of greatest match to the current (residual) vector. 3. Record them with weights. 4. Subtract their contribution from the residual. 5. Repeat until most of the meaning is removed.
A single note-embedding is already a lossy centroid — it superimposes everything the page is about into one point. This loop un-superimposes it, recovering structure the embedding flattened. Two design points are still to be defined:
• Match (TBD): select atoms by cosine (direction is what names a concept), but weight each by its projection magnitude — that magnitude is the concept's salience, the exponent (p^3 = the page is about it; p^1 = a passing mention). Match against the residual after the first pass, not the original. Dictionary choice is the real fork: extract against an open vocabulary (expressive, but fragments near-synonyms into competing atoms), then snap each extracted direction to the nearest entry in the canonical concept lattice maintained for Layer-2 anyway. Open extraction, canonical labelling.
• Subtract (TBD): subtract the projection of the residual onto the chosen atoms, not the raw centroid of the matched words (whose length says nothing about how much of that meaning is present). Cheap grade: project onto the centroid direction and subtract (plain matching pursuit). Proper grade: project onto the subspace spanned by the 2–3 atoms and subtract (orthogonal matching pursuit), which stops you re-picking an atom you have already partly removed — and embedding atoms are never orthogonal, so this matters.
Stopping rule: halt when ‖residual‖/‖note‖ falls below a fraction. That fraction is the coarse/fine dial — low gives a few broad concepts per note, high gives many fine ones. Same chapters-vs-paragraphs dial as Layer 1, now on the concept axis.
• Relatedness becomes GCD. Two notes are related because they share factors — and you can name the shared factors ("both have optimistic-concurrency"). A vector store says two pages are 0.83 similar; a factor index says they share exactly these three concepts. One has a because; the other does not.
• Compile, don't re-derive. The fuzzy RAG-like work — reading a page and naming its concepts — happens once, at ingest. Retrieval through the resulting concept → notes index is then plain lookup. RAG builds the index; it does not answer the query. This is Karpathy's compile-vs-retrieve distinction (LLM-WIKI.md): curate the edge once, traverse it forever.
The strongest single frame for the write-up: you can always see why a note was fetched — it was named, it was linked, or it shares a labelled concept. Vector search returns a ranked list with no reason attached. An embedding hit has no because; that is the one advantage similarity search structurally cannot answer.
But state the tiers honestly, because the middle one is only semi-legible (a label approximates a direction): curated links (fully legible, a human asserted the relevance) > concept factorisation (legibly approximate — labelled directions, weighted, has a because) > raw vector similarity (opaque). This adds a middle tier; it does not displace the top one.
The decomposition is matching pursuit / orthogonal matching pursuit — signal-processing lineage. It is the same sparse-decomposition move as the spectrogram / Fourier work, with the dictionary being word-embeddings instead of sinusoids. Well-understood method; reads as principled rather than hand-wavy.
The concept-index structure is formal concept analysis (a lattice between objects=notes and attributes=concepts, via a Galois connection). The honest consequence: concepts are not independent primes — optimistic-concurrency implies concurrency, so there is a partial order among the factors. The concept space is a lattice, not a flat bag, which is precisely why the concept index can itself be hierarchical — Layer 1's tree discipline, applied to ideas instead of topics.
• Canonical primes. Unique factorisation needs canonical concept names. If concurrency, optimistic-locking and if_rev fragment one prime into three, the index rots silently — the same silent-failure risk as a missing link, relocated into the vocabulary. The trade is honest, not free: pay it once at ingest with a controlled concept vocabulary instead of paying an embedding model every query.
• Discovery of the unknown-unlinked. Navigation presupposes an entry point. If you do not know the key and nothing where you stand links to the target, you cannot traverse to it — and finding what you did not know existed is exactly RAG's real job. So the end state is not "no RAG": it is RAG demoted from primary access to a discovery fallback for the unlinked frontier.
• Scale crossover. Mnemonic keys and curated hubs are wonderful at hundreds of notes and one or two hops from an index. At large N the hubs bloat and the indices themselves want search. Name which end of that curve the system is on rather than claiming the approach is scale-free.
• structured-context-protocol — the retrieval-by-navigation and compile-vs-retrieve thesis this extends
• papers/karpathy-llm-wiki — compile-vs-retrieve; ingest/query/lint primitives
• TRIGGERS — hand-built concept → note router; the manual prototype of Layer 2
• README/orphaning — link hygiene is retrieval quality; a missing link is a silent retrieval failure
• gdata-server / JSONHTL_SPEC — the hypertext substrate the concept index would be stored in and served from
• envoy/states/gathering and envoy/design-phases — retrieval-by-navigation in action; the concept index would give the gathering phase a second lookup layer. This idea feeds the Envoy POC / Medium write-up.
• Pin the two TBDs concretely: metric (cosine-select / projection-weight), dictionary (open-extract → snap-to-canonical), subtraction grade (MP vs OMP), residual threshold.
• Prototype: factor a handful of existing notes and eyeball whether the extracted concepts are sensible.
• Seed a canonical concept vocabulary / lattice from the current note corpus.
• Decide how the concept → notes index is stored and queried within JSONHTL / gdata-server.
• Layer 1 is separable and can start independently: restructure CONTENTS into a thin top index over section/chapter indices.