Performance & Future-Proofing — Research Findings and Technical Spec

Status: DRAFT for review Date: 2026-09-27 (first draft 2026-09-15) Baseline: v4.3.3 (main @ 8ceda53); every finding re-verified against it on 2026-09-27. Items fixed on main between the two dates are marked RESOLVED and kept for the record. Companion: PERFORMANCE-FUTURE-PROOFING-PLAN.md (phased delivery plan) Supersedes / extends: SPEC_PERFORMANCE_IMPROVEMENTS_v3.12.0.md (repo root) and SESSION_PROMPT_PERFORMANCE_FIXES.md, which cover retrieval quality on one prose corpus. This document covers the whole runtime: hooks, CLI, MCP, daemon, build, synapse store, dependencies, and measurement.

Scope note. CLAUDE.md routes BRD/TRD strategy documents to the private neuralmind-marketing repo. This is an engineering spec that references file:line in this repo and feeds docs/specs/ like the two benchmark specs already here, so it lives here. Move it if you prefer.

0. Executive summary

NeuralMind is mature: the retrieval pipeline, incremental graph extraction, turbovec quarantine/recovery, hook fail-open discipline, and the claims provenance guard are all genuinely well engineered. The audit found that the remaining performance problems are architectural, not algorithmic — the same expensive work is repeated in the wrong process at the wrong time — plus a handful of shipped-broken or drifting items that a future-proofing pass must clear first.

The six findings that matter most, in priority order:

The architectural answer (Section 5) is to make per-prompt processes read-only and cheap, move all index maintenance into one warm, idle-expiring daemon that hooks and MCP share, and stamp every persisted artifact with the version of the code and model that produced it. Everything else in this document is a constant-factor improvement layered on that.

1. Method

1. Static audit of the hot paths by five parallel read-throughs (retrieval, synapse store, process startup/hooks/MCP/daemon, build/indexing, measurement/CI), each producing file:line evidence. Headline claims were independently re-verified in source before inclusion here.
2. Reproduction where a claim was cheap to prove: the MCP 2.x crash was reproduced in a scratch venv; import cost was measured with python -X importtime.
3. External research on the dependency surface (MCP SDK, turbovec, ONNX Runtime, py-tree-sitter, CPython free-threading, embedded vector stores, small embedding models, Claude Code hook contract). Sources in Section 9.
4. Cross-check against the repo’s own performance notes (SPEC_PERFORMANCE_IMPROVEMENTS_v3.12.0.md, SESSION_PROMPT_PERFORMANCE_FIXES.md, tests/benchmark/peptide_report.md, docs/HONEST-ASSESSMENT.md).

Nothing in this document is an invented figure. Where a number is a projection it is labelled as one.

1.1 Measurements taken on this checkout

Linux x86_64 container, Python 3.11.15, clean venv resolving to mcp 2.2.0, onnxruntime 1.30.0, turbovec 1.0.0, tree-sitter 0.26.0, numpy 2.4.6. Wall-clock via date +%s%N around the subprocess, 3–5 runs, values rounded. Fixture = scratch copy of tests/fixtures/sample_project; synthetic = bench/benchmark_turbovec.generate_synthetic_repo(n_files=600). Fixture runs used NEURALMIND_ORT_THREADS=1; the synthetic cold build used the default thread count.

Re-measured on v4.3.3 (8ceda53), 2026-09-27, same container and venv method, fixture only:

Two things to read off this table. First, the hook’s cost is the no-op build plus an ONNX session, and both scale with the repo, so a real 50k-node repo will sit well above Claude Code’s comfort zone on every prompt. Second, the edge growth is specific to markdown contains edges (B2), which is why the markdown-free synthetic corpus is stable.

2. What is already good (do not re-propose)

This list exists so reviewers and future implementers don’t spend budget re-solving solved problems.

• Lazy heavy imports. numpy, onnxruntime, chromadb, mcp, pydantic, cryptography and tree-sitter are all deferred; import neuralmind.cli pulls only yaml + sqlite3 from third parties (measured ≈90 ms warm in a clean venv with the full vector stack installed; bare interpreter ≈11 ms). CI asserts the default install is ChromaDB-free (ci.yml:167-186).
• Incremental graph extraction is real. Content-hash + mtime cache, persisted importer index, transitive-importer invalidation, fail-open on a corrupt cache, and a test proving incremental output equals a full rebuild (graphgen.py:1170-1391, incremental_extract.py, tests/test_graphgen.py:1172).
• Community-ID carry-over keeps unchanged nodes byte-identical so the embedder’s content-hash skip actually fires (graphgen.py:1093-1137).
• turbovec quarantine + rebuild from persisted documents (turbovec_backend.py:393-471) — a backend major bump is recoverable, not fatal.
• IR versioning with an explicit unsupported-version error (ir.py:39-46, 481-502).
• Synapse store basics: WAL, synchronous=NORMAL, 30 s busy timeout, indexed spreading activation that never loads the full edge table, executemany on the batched writes, a rollback-safe tested v0→v1 migration, and a node_a < node_b CHECK constraint.
• Hooks fail open with per-feature kill switches (NEURALMIND_BYPASS, NEURALMIND_SYNAPSE_INJECT, NEURALMIND_REUSE_FEEDBACK) and stdout kept clean for the JSON channel.
• MCP server loads each project once per process and runs tool work in asyncio.to_thread so the event loop never blocks (mcp_server.py:48, 1292).
• Daemon has a transport-free dispatch() seam, per-project RLock, 0600 token file, Windows-correct PID liveness, job eviction and graceful SIGTERM (daemon.py). It is inert only because nothing starts it.
• Secret-scan prefilter with measured justification and size caps (secret_scan.py:479-543) — the best-engineered hot path in the codebase.
• Claims provenance guard (site/claims.json, tests/test_site_claims.py, tests/test_docs_claims.py) — this is why no latency number has been published without a harness. Keep it; the plan adds the harness.
• Bounded-cost discipline in several places: betweenness sampling above 2,000 nodes, doc↔code coupling capped at 50 edges, replay queue capped at 1,000, team bundle capped at 5,000 entries, _displace budget-neutral swaps with a documented rationale.

3. Detailed findings

Each finding has an ID used by the plan. Severity: P0 ship-blocking or user-visible today; P1 materially affects large repos or long-lived installs; P2 hygiene / future risk.

3.1 Process model and hooks

3.2 Retrieval hot path

3.3 Build and indexing

3.4 Synapse store

3.5 Dependencies, platform, versioning

3.6 Measurement and CI

4. External research — what the ecosystem looks like in September 2026

Findings that change what “future-proof” means for this codebase.

MCP Python SDK. 2.0.0 shipped 2026-07-28 as “a major rework”; latest is 2.2.0 (1.30.0 on the 1.x line, 2026-09-07). The low-level Server now takes on_list_tools= / on_call_tool= constructor kwargs with (ctx, params) -> Result handlers returning ListToolsResult / CallToolResult; automatic JSON-schema input validation was removed. The migration is ~15 lines here because TOOLS is plain dicts and handle_tool_call is a sync dict dispatch (mcp_server.py:1150-1272). The 1.x line is still receiving releases, so a dual-path shim is cheap and lets the cap become <4.

turbovec. PyPI classifier Development Status 3 – Alpha; 1.0.0 on 2026-08-18; wheels for manylinux_2_28 x86_64/aarch64, macOS arm64, win_amd64 only. This is the default backend for every user. The recovery path for format breaks exists; what does not exist is protection against a behavioural regression (ranking, quantisation error) shipping straight into the default install. The plan adds a nightly canary and an upper cap on the major (<2) with an explicit re-evaluation trigger, and keeps ChromaDB as the supported fallback rather than the “legacy” one.

ONNX Runtime. Session creation cost is dominated by online graph optimisation; the documented mitigation is offline optimisation (serialise the optimised graph once, load with optimisations disabled) or the ORT model format with saved runtime optimisations. 1.24.x ships Python 3.13/3.14 wheels; the project announced it stops publishing 3.10 wheels after 1.23.

Embedding models. all-MiniLM-L6-v2 remains the right default at this size (≈46 MB, 384-d, ~15 ms/1k tokens on CPU). Two directions worth an experiment, not a default change: (a) Model2Vec static embeddings (potion-base-32M: ~70× faster than MiniLM on CPU at ~92 % of its retrieval quality, no ONNX session at all, so hooks could embed a prompt in-process in single-digit ms); (b) EmbeddingGemma-300M as an opt-in higher-quality model for code (best MTEB-Code score in its size class, but ~620 MB — an opt-in like the existing bge-large path). Either requires D6 (model identity in the index) first.

py-tree-sitter. 0.25/0.26 removed Language.version (→ abi_version), Language.query() (→ Query(language, source)), and keep_text. Grammar packages ≥0.23 return capsules that older cores cannot accept. The code already targets the capsule form; the floor must say so. _make_parser already handles the Parser churn.

CPython. 3.10 EOL October 2026. Free-threaded builds are officially supported in 3.14 (5–10 % single-thread overhead; C extensions without the free-threading flag silently re-enable the GIL). The relevant risk for this codebase is not speed but correctness of unlocked shared state under a free-threaded interpreter; the plan adds a 3.14t CI leg as informational first.

Embedded vector search. sqlite-vec (exact brute-force, fine to low millions), usearch (HNSW), and binary/int8 quantisation are all viable in-process options; none is obviously better than turbovec for this workload today. The right investment is the backend contract (a versioned VectorBackend protocol with parity tests) so any of them can be swapped in without touching the selector, rather than another migration.

Claude Code hooks. Command-hook default timeout 600 s; UserPromptSubmit 30 s; SessionEnd 1.5 s shared budget; async: true and asyncRewake: true exist for command hooks; the docs recommend the if field to avoid spawn overhead and treat a timed-out hook as non-blocking. New event types since the hooks were written (PostToolBatch, PostCompact, FileChanged, SubagentStart/Stop) are directly relevant: FileChanged can replace the polling watcher for Claude Code users and PostToolBatch can replace per-tool compress-read spawns.

5. Target architecture

5.1 Principle: three process classes, one owner of mutation

Today all three classes run the batch job’s code path. That is finding H1–H3 in one sentence.

5.2 Index freshness without a rebuild

Introduce .neuralmind/index.stamp (JSON, written atomically at the end of every successful batch job):

{
  "stamp_version": 1,
  "graph_sha256": "…",          "graph_schema_version": 3,
  "ir_version": 2,
  "embedding": {"model_id": "all-MiniLM-L6-v2", "model_sha256": "…", "dim": 384, "max_tokens": 256},
  "vector_backend": {"name": "turbovec", "version": "1.0.0", "index_format": 5},
  "bm25": {"version": 2, "tokenizer": "prose"},
  "synapse_schema_version": 1,
  "neuralmind_version": "3.12.0",
  "built_at": 1757980000.0
}

Edge processes and the warm service load if the stamp matches what the running code expects; otherwise they degrade (edge: no output; service: schedule a batch rebuild and serve the old index until it lands). The stamp replaces the mtime heuristic (R10), gives D6 its model identity, and gives B9 its schema gating in one artifact. A model or backend change invalidates by construction instead of silently mixing spaces.

5.3 Split build() into load() and maintain()

• NeuralMind.load() — read stamp, mmap/load vectors, load BM25, open synapse store. No writes. Used by edge and service.
• NeuralMind.maintain(force=False) — the current build() body, plus atomic writes, plus stamp. Used only by batch jobs and by the service’s background rebuild.
• _ensure_built() becomes _ensure_loaded(), and raises GraphNotBuiltError (already exists) instead of building.
• create_mind(auto_build=True) keeps its signature (the query interface must not change — SESSION_PROMPT_PERFORMANCE_FIXES.md rule 5) but auto_build means “maintain if the stamp is missing or stale”, not “always”.

5.4 Hook fast path

neuralmind _hook prompt-submit
  ├─ parse argv[1:3] only (no argparse tree)          ~1 ms
  ├─ read stdin JSON                                  ~0 ms
  ├─ daemon discovery file present & alive?
  │     yes → POST /hook/prompt-submit, 100 ms budget → print → exit 0
  │     no  → spawn daemon detached (once per TTL), fall through
  ├─ stamp present & matches?  no → exit 0 (silent)
  ├─ lexical seeds: top-k node ids by identifier match from a
  │     small persisted `node_names.json` (no ONNX)              ~5 ms
  ├─ SynapseStore.spread(seeds, depth=2, top_k=8)                ~5–20 ms
  └─ print context, exit 0

Registered hook entries gain "timeout": 5 (seconds), and compress-read / edit-activity gain an if predicate so Claude Code skips the spawn for tools that cannot produce a hit. Where the client supports PostToolBatch, register one batch hook instead of three per-tool hooks. Where it supports FileChanged, prefer it over the polling watcher.

5.5 Synapse store hardening

• All write transactions BEGIN IMMEDIATE; one _write_tx() wrapper with bounded exponential-backoff retry on OperationalError: database is locked.
• Co-activation batch cap: MAX_COACTIVATION_NODES = 64 (env-tunable). Above the cap, keep the top-scored 64 by file relevance and record the rest as activation counts only. Reject batches early with a debug log.
• decay() gated by last_decay (skip if < 6 h) and executed only by batch jobs / the service; hooks request a decay via a meta flag.
• Rebuild-time remap + GC: after maintain(), join synapses against the live node set; migrate edges whose IDs changed only by the line-index suffix (markdown headings) using a (file, heading-slug) secondary key; delete edges whose nodes are gone (with a 30-day grace via last_seen).
• Immortality fix: LTP floor applies only while both endpoints are live; a PRUNE_ORPHANS pass ignores the floor.
• Stable IDs for markdown headings: {file_id}__h_{slug(heading)}[_{n}] with n disambiguating duplicates, no line number.
• Add index idx_syn_ns_last (namespace, last_activated). Push merge/ORDER BY/LIMIT for edges()/transitions() into SQL.
• One connection per store instance (thread-local), PRAGMAs once, schema executescript guarded by the meta.schema_version check.
• Periodic PRAGMA wal_checkpoint(TRUNCATE) + PRAGMA incremental_vacuum in the batch decay pass; auto_vacuum=INCREMENTAL set on creation (v2 migration for existing DBs).
• Single ordered migration list keyed off meta.schema_version, forward guard (recorded > SCHEMA_VERSION → open read-only, warn once).
• Replay queue: batched insert + periodic trim. Pending-review queue: a table.

5.6 Build path

• One filesystem walk producing {suffix: [paths]} with ignore-pruned os.scandir; .neuralmindignore parsed once.
• Extend the content-hash cache to _DOC_SUFFIXES + _SCHEMA_SUFFIXES.
• add_edge dedup on (relation, source, target, source_location); a test asserting len(links) is stable across three no-op incremental builds (this doubles as the B2 reproduction).
• Atomic writes (tmp + os.replace) for every artifact; drop indent for machine-read files; content_text for markdown moved out of graph.json into the store’s document column (it is already persisted there).
• Chunked embedding (2,048 texts per chunk, persist per chunk); dynamic padding to per-batch max length; _BATCH 64; optional int8 model behind NEURALMIND_EMBED_QUANT=int8.
• ProcessPoolExecutor for pass-1 symbol extraction across files of one language (NEURALMIND_BUILD_WORKERS, default min(4, cpu_count)); pass 2 stays single-process.
• File-size cap 2 MB with a logged skip; Louvain skipped when changed_set is empty and no new files.
• extraction_cache.json carries schema_version; mismatch → full rebuild.

5.7 Retrieval

• BM25 posting lists (term → [(doc_idx, tf)]), persisted with tokenizer and mode in the payload header; loaded lazily; BM25 build skipped when added + updated == 0. Code projects either index code nodes too or skip the build entirely.
• turbovec search: batch the hit fetch (WHERE uid IN (…)), drop the COUNT(*).
• Community summaries: precomputed once per load into a dict.
• Prose cross-chapter expansion over an adjacency map, not the edge list.
• Config surface for tuned weights: neuralmind-retrieval.yaml (or the existing .neuralmind.yaml) with the current values as defaults and the corpus they were tuned on recorded in a comment; domain tables (terminology.py, chapter intent weights) become optional plug-ins loaded from the project, not the library.
• Fix the docstring/behaviour drift in R8 (either direction, but recorded).

5.8 Dependencies and versioning

• mcp>=1.28.1,<4 with a transport shim: try 2.x constructor kwargs, fall back to 1.x decorators. Add a CI step that actually starts neuralmind-mcp and completes an initialize + tools/list round-trip over stdio.
• turbovec>=0.7,<2 with a documented re-evaluation trigger; nightly canary workflow that installs latest turbovec and runs evals/parity.
• tree-sitter>=0.22; grammar packages >=0.23; _load_language cached.
• __version__ from importlib.metadata.version("neuralmind") with a fallback literal; a test that asserts it equals pyproject.toml.
• CI matrix: add 3.13 (required) and 3.14 + 3.14t (informational, allowed to fail for one release). Drop 3.10 in the release after Python 3.10 EOL. Update docs/COMPATIBILITY.md to match reality.
• mypy becomes required for neuralmind/synapses.py, core.py, hooks.py first (incremental strictness), then the rest.
• Apply slow / integration / perf markers; move test_synapse_latency.py behind perf.

5.9 VectorBackend contract

A versioned typing.Protocol in neuralmind/backends/protocol.py:

class VectorBackend(Protocol):
    backend_id: str            # "turbovec" | "chroma" | …
    format_version: int
    def load(self, stamp: IndexStamp) -> None: ...
    def upsert(self, rows: Iterable[NodeRow]) -> UpsertStats: ...
    def search(self, vector: np.ndarray, k: int, where: Filter | None = None) -> list[Hit]: ...
    def get_nodes(self, ids: Sequence[str]) -> list[NodeRow]: ...
    def community_summaries(self) -> Mapping[int, CommunitySummary]: ...
    def bm25(self) -> BM25Index | None: ...

context_selector.py talks only to the protocol (today it probes six backend-specific attribute names — context_selector.py:580, 608, 963, 1596-1598). evals/parity becomes the conformance suite any new backend must pass.

6. Measurement spec (the first deliverable)

Harness: tests/benchmark/perf.py (stdlib + the package; reuses bench/benchmark_turbovec.py’s spawn isolation, getrusage, _dir_size_bytes). Emits one JSON document per run:

{
  "schema": 1, "commit": "…", "python": "3.12.4", "platform": "linux-x86_64", "runner": "ubuntu-24.04",
  "corpus": {"name": "synthetic-1k", "files": 1000, "nodes": 10412},
  "build": {"cold_s": 0.0, "incremental_noop_s": 0.0, "incremental_1file_s": 0.0, "peak_rss_mb": 0.0},
  "artifacts": {"graph_json_mb": 0.0, "vector_store_mb": 0.0, "bm25_mb": 0.0, "synapse_db_mb": 0.0},
  "cold_start": {"import_cli_ms": 0.0, "load_index_ms": 0.0, "first_query_ms": 0.0},
  "query": {"warm_p50_ms": 0.0, "warm_p95_ms": 0.0, "n": 50},
  "hook": {"prompt_submit_p95_ms": 0.0, "compress_read_p95_ms": 0.0, "edit_activity_p95_ms": 0.0},
  "mcp": {"tools_list_ms": 0.0, "query_tool_p95_ms": 0.0},
  "embed": {"vectors_per_s": 0.0},
  "synapse": {"reinforce_200files_ms": 0.0, "decay_ms": 0.0, "db_growth_after_100_sessions_mb": 0.0}
}

Corpora: synthetic-100, synthetic-1k, synthetic-10k (generated by generate_synthetic_repo, deterministic seed) for the scaling sweep; the existing tests/fixtures/sample_project for the CI gate; the four pinned OSS repos from evals/public for the reproducible-on-demand table.

Gating rules (added to ci-benchmark.yml):

Baselines: tests/benchmark/perf_baseline.json committed and updated by an explicit chore(perf): refresh baseline commit only. The regression test never reads a gitignored results file (M3).

Instrumentation: RetrievalTrace gains timings_ms: dict[str, float] (embed, vector_search, bm25, synapse_spread, assemble); hooks.py and mcp_server.py entrypoints wrap in a perf_counter span that is appended to the existing .neuralmind/metrics/*.jsonl (already has 30-day retention). neuralmind stats --perf reads it.

Publishing: every gated number that the site wants to quote goes into site/claims.json with evidence: "ci-gated" per the existing rule.

7. Acceptance criteria for the programme

8. Open decisions for the reviewer

1. Daemon auto-spawn by default? It gives the best hook latency but adds a background process users didn’t ask for. Recommendation: on by default with an idle TTL of 30 min and NEURALMIND_DAEMON=0 to disable; document it in the release notes and neuralmind doctor.
2. Drop Python 3.10 in v3.13 or v4.0? Recommendation: announce in v3.13 release notes, drop in the first release after 2026-10-04 (upstream EOL).
3. Cap turbovec at <2? Recommendation: yes, with the canary; the quarantine path covers format breaks but not ranking regressions.
4. Model2Vec experiment for hook-side embedding — worth a two-day spike behind a flag once the stamp (D6) lands? Recommendation: yes, as an informational benchmark row, not a default change.
5. Should the domain tables (terminology.py, chapter intent weights) move out of the library into a per-project plug-in now, or when the second prose customer appears? Recommendation: now, because they run on every prose query for every user and are the clearest example of D7.
6. Where do these two documents live? docs/specs/ here (current) or neuralmind-marketing/internal/plans/ per CLAUDE.md.

9. Sources

Internal:

• SPEC_PERFORMANCE_IMPROVEMENTS_v3.12.0.md, SESSION_PROMPT_PERFORMANCE_FIXES.md
• tests/benchmark/peptide_report.md, tests/benchmark/retrieval_report.md, tests/benchmark/ADVERSARIAL_QA_ANALYSIS.md
• docs/HONEST-ASSESSMENT.md (“What we haven’t measured well yet”), benchmarks/README.md
• site/claims.json (unsourced_do_not_use), docs/COMPATIBILITY.md, ROADMAP.md, KANBAN.md

External (accessed 2026-09-15):

• MCP Python SDK migration guide — https://py.sdk.modelcontextprotocol.io/migration/ ; PyPI mcp release history (2.0.0 2026-07-28, 2.2.0 latest, 1.30.0 2026-09-07) — https://pypi.org/project/mcp/
• Representative downstream breakages from the same cap mistake: chunkhound#405, mcp-pandoc#40, mcp-logseq#92, nutrient-pdf-mcp-server#9
• turbovec on PyPI (Alpha classifier, 1.0.0 2026-08-18, wheel tags) — https://pypi.org/project/turbovec/ ; https://github.com/RyanCodrai/turbovec
• ONNX Runtime graph optimisations / offline mode — https://onnxruntime.ai/docs/performance/model-optimizations/graph-optimizations.html ; ORT-format runtime optimisation — https://onnxruntime.ai/docs/performance/model-optimizations/ort-format-model-runtime-optimization.html ; releases (3.13/3.14 wheels; 3.10 wheel sunset) — https://github.com/microsoft/onnxruntime/releases
• py-tree-sitter 0.25/0.26 API changes — https://tree-sitter.github.io/py-tree-sitter/ ; tree-sitter-python#280 (capsule/core version mismatch)
• Python free-threading HOWTO — https://docs.python.org/3/howto/free-threading-python.html
• Model2Vec — https://github.com/MinishLab/model2vec ; EmbeddingGemma-300M ONNX — https://huggingface.co/onnx-community/embeddinggemma-300m-ONNX
• sqlite-vec — https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/index.html
• Claude Code hooks reference (timeouts, async, if, event list) — https://code.claude.com/docs/en/hooks
• Chroma 1.x migration notes — https://docs.trychroma.com/docs/overview/migration