Gloss — Vector dimension sweep, 18 September 2026

Parent: gloss. Recommendation: retain 1,536 dimensions for now. Shortening changes many residual-selected labels and does not consistently resolve unsupported specifics. This is a stability experiment, not proof that 1,536 is semantically optimal.

Ran all 712 non-empty notes from the fixed 713-key public snapshot against the same 14,554 filtered atoms. One empty note is recorded as excluded. Four concurrent processes, four numerical-library threads each, using the 16 available CPUs; 5.72s wall time. Zero API calls. Each note and atom vector was prefix-truncated and renormalized. Existing chunk pooling was held fixed before truncation for the 11 long notes.

Primary comparison excludes three Gloss write-ups (709 notes). Percentages compare each size with the existing 1,536-dimensional result. Label retention is the mean fraction of reference labels retained, ignoring order. These are agreement figures, not accuracy scores.

DimensionsSame first labelSame complete label setPursuit label retentionDirect-cosine label retentionMean residual norm ratio
25657.3%4.7%39.6%58.2%0.7531
51278.4%22.8%60.2%75.5%0.7747
102487.2%50.1%76.9%88.7%0.7847
1536100.0%100.0%100.0%100.0%0.7950

Removing 105 repetitive drain logs leaves 604 notes: first-label agreement at 256/512/1,024 dimensions is 52.0%/75.2%/85.8%; pursuit label retention is 37.8%/58.5%/77.0%. The main conclusion survives this sensitivity check. Complete subgroup statistics are in summary.json.

Smaller spaces give lower residual norms, but norms across different dimensional spaces are not a common semantic-quality score. Direct-cosine labels are more stable than residual pursuit: at 1,024 dimensions they retain 88.7% of reference labels versus 76.9% for pursuit.

Source-checked examples show mixed effects. At 512 and 1,024 dimensions, the Python exception policy loses the unsupported Perl label. At 512, the Kathy poem loses the unsupported summer label, and the Kombu teardown gains decommissioning. Conversely, at 256 the eclipse note gains the unrelated David Lloyd schedule, and the terminated Kombu instance gains ongoing. Therefore label changes cannot all be called regressions or improvements.

The table below compares the same 25 examples used in the corpus review. It is a qualitative inspection sample, not blind scoring or a corpus accuracy estimate. No new aggregate quality verdict is claimed.

Note256 dimensions512 dimensions1024 dimensions1536 dimensions
PROGRAMMING_RULES/philosophycorrectness with authority; philosophies; worst casecorrectness notes; philosophies; arbitrary pythoncorrectness notes; philosophies; arbitrary pythoncorrectness notes; philosophies; arbitrary python
PROGRAMMING_RULES/exceptionsexception handling; core principle; perl pythonexception handling; python api; general principlesexception handling; python api; general principlesexception handling; perl python; general principles
ideas/concept-factorisationconcept notes map; storage architecture; concept factorisation hierarchicalconcept notes; core metadata infrastructure; concept factorisation hierarchicalconcept notes; indexing; concept factorisation hierarchicalconcept notes; indexing; concept factorisation hierarchical
first-thoughts/representation-not-languageopen questions; structurally impossible; map and territoryopen questions weak; variant; note embeddingsopen questions weak; variant; note embeddingsopen questions weak; note embeddings; behavioural equivalence
gdata-server/notes/designgdata notes; imap client module; architecture notesgdata notes; imap client module; architecture notesgdata notes; notes client; python modulegdata notes; notes client; python module
envoy/design-phasesimplementation notes; solver control structure; email processing pipelineimplementation notes; finite state; email processing pipelineimplementation notes; finite state; gathering summarising workingimplementation notes; processing state machine; envoyresponse schema
README/orphaningparent notes; dead link recovery; creating and updatingavoiding orphaned notes; metadata and links; article revision issuesavoiding orphaned notes; metadata and links; blueprint page checklistavoiding orphaned notes; metadata and links; design documentation
TFG/exercise-demo-protocolfunctional exercise; testing checklist; summarising working codingfunctional exercise; review and handover; sample demonstrationsfunctional exercise; review and handover; sample demonstrationsfunctional exercise; review and handover; sample demonstrations
using-gps-datagazetteer point; implementation details; ground truthchecks gazetteer comparisons; ground truth; jsonhtl noteschecks gazetteer comparisons; project notes; answered nearest stationchecks gazetteer comparisons; project notes; answered nearest station
popit3/opscontext restored; maildir test completion; triage gatheringmaildir test completion; computer crash occurred; popsmaildir test completion; computer crash occurred; popsmaildir test completion; computer crash occurred; pops
pwsafe/transport/planapi implementation plan; status complete; open questions weakphase webdav plugin; transport infrastructure; pwsafe article revisionwebdav transport; pwsafe article revision; phase designpwsafe article revision; transport plugin; current phase
Z80-sudoku-challenge/code/deterministic-fill-loopdeterministic fill loop; lib dap; board base addressdeterministic fill loop; address of cell; sudoku boarddeterministic fill loop; address of cell; sudoku boarddeterministic fill loop; address of cell; bit candidate mask
delayrepay/session-2026-08-29-kombu-decommissionansible on kelp; ongoing; firefox vncfirefox vnc; ansible on kelp; decommissioningfirefox vnc; writeup; ansible on kelpfirefox vnc; kombu; automated via ansible
ask/jobs/schemajson schema; cover letters; calls openaijson schema; cover letter angle; calls openaijson schema; cover letter angle; docx generationjson schema; cover letters; docx generation
location-db/outcodes-1degcardiff gloucester swindon; postcodes; gwml neutralbristol bath newport; postcodes; gwr delaybristol bath newport; postcodes; gwr delaybristol bath newport; postcodes; cardiff gloucester swindon
personal/interestsinterests hobbies; johnsrcritchley gmail; GPSinterests hobbies; GPS; john has askedinterests hobbies; GPS; john has askedinterests hobbies; GPS; john has asked
proposals/section-editingstructural edits; new jsonhtl block; gdata notesstructural edits; new jsonhtl block; orphaned note subtreesjsonhtl documentation improvements; blocks section; semantic specjsonhtl documentation improvements; completed section; semantic spec
astronomy/eclipse-2026path of totality; swindon reading; david lloyd schedulepath of totality; bristol london; solar eclipsepath of totality; bristol london; solar eclipsepath of totality; bristol london; solar eclipse
poems/first-to-Kathymoments; kathy; sonnetkathy; poems; lovekathy; summer poem; lovekathy; summer poem; forever
uucp/setup-statustaylor uucp; protocol restrictions; correctness and authorityroles uucp; stunnel client config; jsonhtl documentation improvementsroles uucp; stunnel client config; jsonhtl documentation improvementstaylor uucp; existing stunnel; protocol restrictions
ansiblegit ansible; comprehensive notes; playbooksgit ansible; private notes; ephemeral hostsgit ansible; private notes store; ephemeral hostsgit ansible; private notes store; hostnames
music/exercise-folk-rockexercise music; functional exercise; fetch notesexercise music; functional exercise; shantiesexercise music; functional exercise; shantiesexercise music; functional exercise; shanties
hosts/gravlaxwebdav notes; deployed to gravlax; freebsd hostwebdav notes; deployed to gravlax; freebsd hostwebdav notes; kelp gravlax; freebsd hostwebdav notes; gravlax and kelp; freebsd host
fitness/bodypump-attendancebodypump playlist importer; assess in august; david lloyd schedulebodypump; david lloyd schedule; accommodation figuresbodypump; david lloyd schedule; costs september octoberbodypump; david lloyd schedule; bristol minima shown
ssl/certhub-tododbcerts; future improvements; certhubcertificate configuration; future work; certhub pollingdbcerts; certhub; process improvementscertificate configuration; certhub; future work

Storage and arithmetic scale approximately with dimensions: 256/512/1,024 use one sixth/one third/two thirds of the vector storage. The full float64 note-plus-atom matrix is only 179 MiB. Concurrent single-run timings are not a controlled speed benchmark. There is little pressure to sacrifice stability here.

Validation: all 15 unit tests passed; full-width labels and direct-cosine ordering match the previous corpus run, with coefficients and residuals within 1e-10. Provenance hashes, worker thread counts and timings are in run.json. All 713 keys are represented in comparison.csv; complete numerical results are in results.json.

Files: /home/john/py/gloss/artifacts/dimensions-2026-09-18/. Reproduce offline: python dimension_sweep.py --processors 16 --output artifacts/dimensions-new-run. No virtual environment or package installation was used for this sweep.

version 1  ·  created 2026-09-18  ·  updated 2026-09-18  ·  tags gloss, corpus, embeddings, results