Parent: gloss. This turns the experiment ranking into an implementable first version. It deliberately keeps generated labels out of source-note prose until they have passed review.
At ingest, extract the note text exactly as the corpus pipeline does, use the filtered 14,554-atom vocabulary and the cached text-embedding-3-small 1,536-dimensional vectors, rank direct cosine matches, and retain the top two labels. Store an evidence status beside each label: text-supported when its terms occur in the source, related when the embedding match is useful but exact wording is absent, and unsupported when a specific claim has no source evidence. The evidence gate is a confidence signal, not a deletion rule.
For a source key such as hosts/gravlax create hosts/gravlax!gloss. The companion note is generated, versioned and disposable; it contains the source key/hash, vocabulary/model versions, two labels, scores, evidence status and source spans. A reserved !gloss suffix keeps it beside the source note while making it mechanically identifiable. Corpus ingestion must exclude keys ending in !gloss from ordinary note embeddings and vocabulary learning. Do not add a backlink to the source note until the companion format has been reviewed.
| Component | Purpose | Initial form |
|---|---|---|
| Canonical concept vocabulary | Stop “webdav vhost”, “webdav notes” and similar variants becoming separate index concepts; preserve aliases and specificity. | A versioned JSON/SQLite table: concept_id, preferred label, aliases, type, parent, status, evidence policy. Seed manually from the filtered candidates and reviewed outputs. |
| Per-note factor record | Make each extracted label inspectable and reproducible. | Companion note at <source-key>!gloss, keyed by source key and source SHA-256: model, dimensions, vocabulary version, labels, weights/cosines, evidence status, matched source spans, created timestamp. |
| Concept-to-note index | Make the factorisation useful for navigation. | The companion notes themselves are the first index: predictable keys let tools read <concept source>!gloss. Add a generated concept hub or query index only when direct key scans become costly. This is the Layer 2 concept index from the design note. |
| Evidence rules | Prevent specific unsupported claims being presented as facts. | Small rule table for dates, places, people, tools, hosts/OSes, statuses and occasions: exact mention required for factual output; related labels remain retrieval-only. |
| Evaluation set | Keep improvements measurable rather than anecdotal. | Freeze the reviewed 25 notes plus the seven checking notes, with source-grounded supported/related/unsupported labels and missed facets. Add examples whenever a production false match is found. |
1. Follow explicit links and the hierarchical index first. 2. Look up exact canonical concepts in the concept-to-note index. 3. Use the two generated labels to expand an unknown frontier, showing each label, weight, evidence status and source passage. 4. Fall back to raw vector similarity only when no labelled route is available. This preserves the notes project’s navigation-first design while giving it a scalable concept index.
When a note changes, recompute only that note, replace its sidecar factor record and update the affected concept postings. Keep the source SHA-256 and vocabulary/model versions so stale factors are detectable. When a canonical concept is renamed or merged, update aliases and postings in one versioned migration; do not silently rewrite historical results.
Do not switch to the large model solely because it has 2,048 or 3,072 dimensions; the review did not establish a quality gain. Do not make matching-pursuit subtraction the only output; direct cosine is the safer default and two-step pursuit can remain an optional diversity view. Do not write generated labels into the notes themselves before the evaluation set and canonical vocabulary are stable. Do not claim that a residual norm measures recovered meaning.
Build companion notes and a predictable-key concept index for the existing 712-note snapshot, with the 25+7 reviewed notes as an audit view. Expose a read-only query that returns the note key, canonical concept, rank/weight, evidence status and source span. Then evaluate retrieval by shared concepts before adding any automated parent/index edits. The only genuinely new human work is canonicalising the first few hundred useful concepts and recording evidence rules; the embeddings and corpus are already cached.
Related design: Concept Factorisation & Hierarchical Indices. Ranking evidence: Experiment ranking; Evidence gate.
Implemented 2026-09-18: gloss_companions.py generated and published 709 companion notes using the <source-key>!gloss convention. Four snapshot entries were skipped: three generated Gloss notes and one empty note. The source notes were not modified; future corpus preparation excludes keys ending in !gloss.