dsh-project-memory

If this plugin saves you an hour of debugging, a star would be appreciated.

English | 简体中文

A persistent project development memory for DeepSeek Harness (dsh) agents. Built specifically for project development, natively integrated with dsh's task system: task lists and files read during a session are automatically persisted as cross-session task records, with tasks ↔ files linked — workflows can be switched and resumed, no need to re-scope the whole project, solving context loss. Underneath sits a hybrid index: document chunks, code symbols, experience notes and insights each form a layer, scored per layer and merged into one ranking. Documents (PDF/Markdown/txt) and code symbols are stored separately per workspace; documents are automatically cross-linked to the code symbols they mention. Experience notes (problem → solution) are automatically deduplicated, preventing repeated mistakes. Data survives session compaction and handover; recalls include path:line citations for source verification.

The plugin keeps a compact project memory on disk, with every entry pointing to a concrete file and line — the agent can reorient quickly instead of re-reading the whole project.

Features

• TaskBridge: cross-session development tasks — the session's todo list and the files it touches are persisted as per-project task entities, so a workflow can be switched and resumed without re-scoping the project. Associated files stay in recency-weighted order (a read never outranks a written file), so a resumed session sees where to look first. New sessions continue with list_tasks → select_task. Work delegated to subagents creates no task. On hosts without session events the task tools still work as a plain record list.
• Task panel and editing — draggable cards show a task's steps and files, and can collapse to a mini-bar or be hidden. Title, steps and status can be edited inline, writing back through the same path as model updates; four visual themes change material, geometry, typeface and density only, and colours follow the host. The panel stays hidden until summoned, syncs in the background on session switch, never reopens itself after a refresh, and a render failure cannot take down the host.
• Document memory — PDF, Markdown and plain text are chunked and summarized without a model call. Each entry keeps a short summary for injection, a bounded term set for recall, and a citation back to the source.
• Code symbol memory — a dependency-free scanner extracts functions, classes, methods, interfaces and type aliases with signatures; covered languages are JavaScript/TypeScript, Python, Go, Rust, C/C++/C#, Java and shell. Where typescript is installed, an optional second layer adds inferred return types, generics and interface shapes; it runs in the background and never blocks indexing.
• Automatic, read-time memorization — with lazy indexing (default), a file is memorized the moment the model reads it, so memory grows as a byproduct of normal work rather than an upfront scan. index_repo or a watched root can cover the rest of the tree; a background poll only re-indexes files that changed.
• Doc ↔ code cross-linking — a document that mentions a symbol is surfaced when that symbol is queried. Links are resolved against the current symbol table, so they cannot go stale.
• Hybrid recall — ranked search over documents, symbols, experience notes and insights, with optional LLM query expansion. The experience/insight layers are re-ordered by cosine when leanvec is present — installed by default, absent ⇒ pure lexical. Tuned for CJK: phrase boost, a synonym table, and CJK-aware boundaries for doc↔symbol linking. Russian (Cyrillic) is supported too: word splitting, a stopword list and Snowball stemming, so Russian queries hit Russian notes and take part in dedup.
• Experience notes — problems → solutions, deduplicated by overlap rather than repeated, bounded by project size, and surfaced both by search and by the same injection path as lessons.
• Tiered insight memory (lessons / decisions / procedures) — one entity across task, project and global scope. Near-duplicates merge or reinforce, and promotion moves an entry between scopes instead of copying it. Use is recorded, so decay and capacity rank by activity rather than age alone. The panel exposes a per-scope view for reviewing and editing entries. That list is ordered newest-first and loaded incrementally: 30 entries at a time, with the next page prefetched before the reader reaches the bottom and off-screen rows skipped by the renderer, so a large memory stays responsive.
• Triggered injection — an insight may carry an authored trigger: only when triggers, guard narrows it, and prevents records what breaks without the entry. Corpus text can never trigger an injection; the statistical channel is gated separately and silent by default. Evidence is graded by source: only host facts (structured tool calls, fs/todo events) and irreversible actions named by the human (commit / publish / release / push / deploy / go public) can inject full text; prose mentions of delete / clean up / render / benchmark only feed the hint channel, and subagent prompts plus notice-shaped messages never count as actions.
• Usage is a skill too — it registers project-memory into the host skills registry (a directory bundle with scripts/pm-digest.mjs, resourceBase pointing inside the package). The model loads the query/write workflow on demand instead of carrying it in every prompt; hosts without a skills service skip it silently.

Installation

The plugin relies only on stable public APIs declared through peerDependencies.

cd dsh-project-memory && dsh plugin --profile web add . -w

The -w (workspace-root) flag is required: the profile directory is a pnpm workspace root, and pnpm rejects add there without it. From elsewhere, use the same command with an absolute path.

The plugin is also published on npm as a scoped package:

dsh plugin --profile web add @yolk_vat-y/dsh-project-memory -w

Each indexed project has its own store at <root>/.dsh-project-memory/. It keeps itself out of git by default — there is nothing to add to your .gitignore. Global-scope entries are kept outside the project in ~/.config/dsh-project-memory/global.json; point insight.globalFile elsewhere to move it.

Usage

The tools below are invoked by the agent, not typed by the user. In the chat, just ask naturally — e.g. "index this project" or "what does the auth module do?" — or simply keep working, and the agent calls the matching tool automatically. By default files are indexed the moment the model reads them, so memory fills in while you work. watch_repo keeps explicitly-watched roots fresh in the background; index_repo forces a full backfill (unchanged files are skipped).

In a git work tree the corpus is whatever git ls-files --cached --others --exclude-standard lists — tracked files plus untracked files that are not ignored — so .gitignore, nested rules included, decides what gets remembered. A directory that is not a git work tree falls back to a plain walk (no ignore rules). Lazy indexing applies the same check when a file first enters the store; a file that is already indexed is refreshed on later reads without asking git again — if .gitignore starts excluding it, the index_repo / watch_repo scan drops it. An explicit index_doc bypasses .gitignore, but still respects roots.exclude, readability and the size limit, and its entry survives later scan passes for as long as the file exists.

/tasks appears in the web / menu inside an icon-bearing Workflow group, with three view entries — Tasks / Project Memory / Global Memory (labelled in the UI language). It is the plugin's only registered command; every other action rides it as a sub-verb driven by card buttons.

How it works

The design follows four principles:

• Volatility — context is ephemeral; it is lost when a session is compacted.
• Persistence — the memory is stored on disk and survives compaction and new sessions.
• Compactness — one bounded declaration line per symbol, and a ≤300-char summary plus a bounded term set per document chunk. Doc→symbol links are not stored; they are computed at read time.
• Verifiability — recalls carry a path:line citation where applicable, so the agent can confirm details against the source.

With default lazy indexing, building the memory does not require an upfront scan: files are memorized as the model reads them, so the memory grows to cover what has been worked with; index_repo or a watched root can cover the rest. Re-reading an unchanged file is a no-op, so the memory stays fresh with minimal ongoing overhead.

Project- and task-scope memory is stored per project and follows the codebase: changed files are re-extracted by content hash, deleted files are removed. Experience notes are ordinary kind: 'experience' entries in insights.json: recallable, deduplicated, and eligible for the same gated hint channel as lessons.

Design

.dsh-project-memory/
  format.json      layout marker (v2, sharded)
  shards/          one self-describing JSON per indexed source file — writes touch only dirty shards
  watch.json       watched roots
  tasks.json       TaskBridge task entities (cross-session)
  binding.json     current session ↔ task binding
  insights.json    project-scope memory (lessons / decisions / procedures / experience notes) — one entity
  vectors.json     optional semantic index (only when the vector layer runs)
  injection-audit.jsonl   one line per real injection (what / why / dropped / budget)
  admission-shadow.jsonl  one line per step, all scored candidates + features (offline replay)

Global-scope entries are not stored under the project: they live in insight.globalFile (default ~/.config/dsh-project-memory/global.json) so every project sees them.

• Incremental — a content hash per file; only changed files are re-extracted.
• Cross-linking — a returned document chunk resolves the symbols it mentions against the current symbol table and appends them as references, so a doc indexed before its symbols still links correctly.
• Query expansion — when enabled, query_memory asks the LLM to rewrite the query into several variants (synonyms, EN/CN, identifier guesses) and merges the scores; when off, queries never touch the LLM. Indexing itself is model-free.
• Consistency — the fact layer follows the codebase (hash re-extract / remove-on-delete); experience entries are updated through the same supersede and forget paths as other insights. Writes take a cross-process directory lock (.write.lock) around a short read-modify-write, so two dsh instances can share one project store: shards converge by indexedAt, a save that cannot get the lock is retried and flushed again at exit, failures stay silent by default (DSH_PROJECT_MEMORY_DEBUG=1 prints one diagnostic line), and entries deleted by another process are not resurrected by a stale in-memory snapshot.

Architecture (Task Panel)

TaskPanel (Container)
├── task-data-store  (server data, cross-tab sync via BroadcastChannel)
├── task-ui-store    (local UI state, localStorage)
├── task-hooks       (useTaskDrag, useTaskEdit)
└── TaskComponents   (MiniBar, TaskCard — presentational only)

The workflow panel is collapsible, adapts to dsh and theme plugin styles, and offers four card styles.

Configuration

Toggling features

The two most relevant switches are lazyIndexing (index a file the moment the model reads it; default on) and autoIndexOnFirstUse (full scan of the current working directory on plugin load; default off). Lazily indexed project roots are automatically registered with the watcher, so changed files stay fresh without an explicit watch_repo.

Root resolution. In order: an explicit root argument; a registered root (watch_repo); the nearest ancestor with a VCS marker (.git/.hg/.svn) or a build/manifest marker (package.json, go.mod, Cargo.toml, pyproject.toml, …); the session working directory, unless it is on the excluded list. A file that matches none of these is not indexed.

When the root comes from the working directory, the model gets one notice per session naming it and how to change it; mute with autoContext.rootNotice: false. Starting dsh in a container directory such as ~/workspace therefore makes that directory the root, and memory spans everything beneath it up to the scan limit — start dsh inside the project for one store per project.

Excluded directories. By default, these are not used as a root, matched exactly (subdirectories are unaffected): the filesystem root, the home directory, temp directories — os.tmpdir() and the shared ones (/tmp, /var/tmp, %TEMP%, %SystemRoot%\Temp) — and system / package-manager prefixes (/opt/homebrew on POSIX; %SystemRoot%, %ProgramFiles%, %ProgramData% on Windows). A session in one of these runs without memory, with one line on stderr.

Custom root lists. The roots block overrides that decision. All three accept ~ / ~/...; non-absolute entries are ignored with a one-time stderr warning (relative paths would resolve against the host cwd and are unpredictable); Windows is case-insensitive:

• roots.allow — allow a directory even when it is on the built-in list, matched exactly. The filesystem root is not eligible.
• roots.deny — this directory cannot be a root, matched exactly; its subdirectories can still be roots of their own.
• roots.exclude — no file in this subtree enters the index, matched by path-boundary prefix (subdirectories included); no new root is inferred inside it either. A complete scan of a parent root skips it and removes entries already indexed under it.

deny only decides whether the directory itself can be a root: when a parent is already a root, files under a denied child are still indexed by that parent — use exclude (or .gitignore, which the git-aware scan already honours) to drop the files too. Explicit index_doc also respects exclude and returns Not indexed for a file under it.

Priority is exclude > deny > allow > built-in list. allowUnsafeRoots: true only releases the built-in list; deny / exclude still win. The same verdict is shared by explicit tool calls, lazy indexing, watch, the session audit trail and TaskBridge. Roots refused by the current policy are dropped from watch.json on startup. Configure under config: of - id: project-memory (CLI and web alike); the web UI has no dedicated settings page, so edit the profile.

Scan limits. One pass covers at most maxScanFiles files (20000) and maxScanDepth directory levels (12). A truncated pass is reported in the index_repo result and logged once per root by the watcher, and it does not remove entries it did not reach. A directory that cannot be read (e.g. EACCES) counts as a truncated pass too and is named in the result: a read failure is not a deletion.

Per-file memory budget. maxFileSizeMb is a memory gate, not a disk-space preference. Indexing a file first materializes it — the whole buffer plus decoded text for code, or every PDF page's text plus the joined markdown for documents — and the peak scales with the file's size. Measured on this machine with npm run bench:peak (peak RSS fitted against disk size over a 1–50 MB ladder, one fresh process per point): ≈10× for typical source code (1 declaration per ~1.4 KB), ≈22× for symbol-dense code (1 per 220 B) and ≈20× for a text-heavy PDF, on top of a ~52 MB process baseline plus a one-off ≈175 MB the first time a PDF is parsed. The 10 MB default therefore keeps a single file at roughly +300 MB RSS for code and +370 MB for a PDF; raising the cap raises that ceiling proportionally.

The store lives in the tree it indexes and ignores itself: a * rule inside the store keeps it out of git status and git add -A, and means git clean -fd leaves it alone. To commit project memory deliberately, git add -f .dsh-project-memory — tracked files are not affected by ignore rules.

Settings live in the plugin's config object. To change them, add an override entry to your profile's cordis.patch.yml — for the web profile that is ~/.dsh/profiles/web/cordis.patch.yml:

- id: project-memory
  config:
    lazyIndexing: true          # on: index files as the model reads them (default)
    autoIndexOnFirstUse: false  # off: no upfront full scan (default)
    llmQueryExpansion: false    # off: do not spend tokens on LLM query expansion (default)
    watch: true                 # on: background refresh for watched roots (default)
    watchInterval: 30           # base poll interval; quiet during a turn, one merged scan at turn end, idle backoff to 2 min
    maxScanFiles: 20000         # per-scan file cap (truncation is reported, never deletes)
    maxScanDepth: 12            # per-scan directory-depth cap
    enableTypeScript: true      # on: type-aware enhancement when TS is installed (default)
    # allowUnsafeRoots: false   # release the built-in list; prefer roots.allow below
    # roots:                     # custom root lists (empty by default)
    #   allow: [/srv/projects/demo]        # exact release of a built-in directory (filesystem root excluded)
    #   deny: [/srv/shared]                # exact: this directory cannot be a root; children still can
    #   exclude: [/srv/monorepo/archive]   # subtree prefix: no file under it enters the index
    # tsPath: /custom/path/to/typescript  # optional: force specific TS install
    # autoContext:              # note: the next two keys live under autoContext, not at the top level
    #   budgetLog: once         # debugging: log budget drops to stderr (default off = silent)
    #   reinjectItemsAfter: 20  # debugging: allow the same insight again after N steps (default 0 = once per session)

Only list the keys you want to change; the rest fall back to the plugin defaults. Verify the result with dsh --profile web --dump-config.

For a one-off run without editing the profile, pass the override as a CLI patch overlay:

dsh web --patch ./config.yml

where config.yml contains the same override block.

Performance

Measured on real projects (2026-10-13)

Four corpora, one Linux machine (Node 24.19), each run twice with the second, warm-cache run quoted. "Cold index" is a full index pass (walk + hash + extract + commit); "query" runs the shipped scorer over 100 sampled queries; "re-index 1 file" is the watch re-index core (read + extract + commit), which excludes the lazy path's root-resolution / .gitignore overhead.

How it scales. Re-indexing a changed file costs O(file), not O(corpus) — 0.3–7.2 ms across every corpus above. Cold load (≈22 µs/file), query (≈0.5 µs/entry on the largest corpus) and resident heap (≈1.5 KB/entry once queried) grow linearly with the index, which keeps small and mid-size projects in the single-digit-millisecond range. Re-reading a file that is already indexed costs ≈0.01 ms: it stops after one stat plus a cached record lookup, with no read, no hash and no git check-ignore.

read+hash depends on the OS page cache — 2.3 s cold vs 0.27 s warm on the 12,634-file tree — so the table quotes the warm figure.

Synthetic Benchmark (Node 24.19, Linux)

Generated code (~4–5 symbols/file), measured 2026-10-13. Reproduce with npm run bench:synthetic -- 5000 (scripts/bench-synthetic.mjs). Pure indexing overhead, no LLM calls. The first query after a write rebuilds the IDF cache (141 ms at 40k entries, 68 ms at 20k, 17 ms at 4k); later queries hit it.

Reproduce it on your own project

The measurement ships with the repository and with the published npm package (scripts/ is part of the tarball). It needs no dsh instance, no network and no model calls, and never touches your project's own store — results go to a temp directory and are removed afterwards:

npm run bench -- /path/to/your/project
# or, with options:
node scripts/bench.mjs /path/to/your/project [--json] [--samples 100] [--no-pdf] [--keep]

It reports the cold index split into read+hash / extract / commit, cold load, IDF rebuild, cold and hot query latency (p50/p95/max over 100 sampled queries through the shipped scorer), single-file hot re-index, store size, resident heap and bytes per entry. Example — the frontend app row above:

cold index   246 ms   (read+hash 8 ms · extract 222 ms · commit 14 ms)   ← 2nd, warm-cache run
store        0.66 MB content · 1.52 MB on disk · 325 bytes/entry · cold load 8 ms
memory       heap 6.3 MB after load → 6.9 MB after the first query (RSS 62 MB)
hot query    p50 0.76 ms · p95 1.4 ms          (2,142 entries)
re-index 1 file  p50 0.3 ms

Pass --queries your-queries.json to run the labeled-set method (hit@5 / hit@10 / MRR) against your own project.

Design tradeoffs

• A short filesystem lock, not an embedded database — each save holds an exclusive .write.lock only for the read-modify-write (parsing stays outside it) and waits synchronously for up to 15 s for a busy peer (correctness comes from acquiring the lock, not from background retries); a write that still cannot land is registered for a synchronous flush when the process exits or the plugin is disposed, without interrupting the user — memory writes are idempotent and cheap to redo; set DSH_PROJECT_MEMORY_DEBUG=1 for a one-line diagnostic. The lock carries pid / host / token: a live same-host holder is never preempted, release verifies ownership, and a crashed same-host lock is reclaimed immediately. Two dsh instances can share one store. No WAL, no daemon, no native dependency.
• Watch: compute outside, commit inside — no lock is held during parsing, and polling with mtime + content hash is used instead of fs.watch: it behaves identically on network drives, Docker volumes and WSL, without duplicate-trigger or missed-event failure modes.
• Corrupt shards are quarantined, not repaired — a shard that fails to parse is renamed and only that file is re-indexed, so there is no WAL or embedded database adding native dependencies, lock contention and failure modes of their own.
• Vectors re-rank, they do not rank — on a paraphrased-query set, pure vector replacement is worse than BM25 (hit@10 53% vs 69%) while weighted fusion helps at small k (hit@3 53% vs 50%), so the optional layer only re-orders the experience/insight layers and never gates. Lexical retrieval still answers the queries this plugin targets: on a real project with 29 labelled queries, file-level hit@5 is 96.6%, and whole-chunk terms raise document term coverage from 27.3% to 100% with MRR unchanged (0.958 vs 0.955).
• Indexing is deterministic and model-free — a model at index time would make two indexings of one document differ, and a wrong translation at query time means zero recall; rules plus symbol links cover the common cases and work offline.
• Model-facing memory: the agent writes, no human in the loop — the consumer is the agent, usually headless, so memory that only promotes when someone clicks a card would never promote at all. draft is a provenance marker plus an evidence threshold, not an approval queue.
• Full entries returned directly — entries are already compact, so returning them whole is more verifiable and one round-trip cheaper.
• forget by query is aggressive; use IDs for precision — experience notes are low-risk and high-volume, so stale noise hurts more than an over-broad delete. For exact deletion use the ID shown by query_memory.
• TypeScript enhancement is optional and lazy — it runs in the background, never blocks indexing, and falls back to the regex scanner; its queue is bounded and deduplicated by content, and any dropped files are reported. The default lib is not loaded: inference that depends on global types (Promise/Array/DOM) degrades to any/unknown, while explicit annotations are unaffected.

Development (for contributors)

These commands are for maintaining the plugin code — regular users do not need them. Installing the plugin only requires the command in Installation.

npm install
npm test                    # 892 tests
npm run eval:injection      # synthetic pool P/R: precision 0.93 / recall 1.00, + a 3-scenario hint-only group
npm run eval:injection -- --store .dsh-project-memory/insights.json   # replay on YOUR store; control group is a hard gate
npm run selfcheck:triggers  # which entries can still push, which declarations are dead (reads your local store)
npm run bench -- /path/to/project   # index/query performance on any project — no dsh needed
npm run bench:peak          # peak-RSS budget of maxFileSizeMb, measured on a 1–50 MB ladder
npm run schema:size         # per-tool fixed prompt cost: chars and a same-scale token estimate

Release notes live in CHANGELOG.md and on GitHub Releases.

License

MIT — see LICENSE.

Copyright (c) 2026 00080000 <3388065969@qq.com>