csift

_{Pronounced "c-sift" (see-sift)}

The missing tool for Claude Code session transcripts.

search, recover, monitor, and audit any Claude Code session straight from the .jsonl logs.

──────────────────────────────────────────────────────────────────────────────
 ❯ what that session decide about rate limiting and wheres the code?
──────────────────────────────────────────────────────────────────────────────

⏺ Bash(csift search "rate limit" @1a2b3c4d -t agent --since 1d)
  matches  1 exchange · 1 session · oldest first

  1a2b3c4d·t42  2026-06-20 22:14:07.811 AEST(UTC+10)
    ▸ agent.message  L8821  Added a sliding-window limiter (10 req/min/IP); the 429 path now
                    returns Retry-After and logs the offending IP — gateway/src/rate_limit.rs:88.
  matched 1 exchange · 1 session · label=agent

That is one regex. The hit comes back as the complete round-trip it belongs to, in output short enough for an agent to read cheaply. No embeddings, database or daemon sit behind it.

Why csift

Claude Code's sessions are plain text in JSONL, so why csift, not grep?

They are plain text. The log holds every prompt, thought, tool call, file edit and pasted image, and every subagent the session spawned. Reading it back is where an agent goes wrong.

• "It deleted my project files, then apologized."

  It was running with --dangerously-skip-permissions. It wiped the files it had built, said sorry and stopped dead. Asked to dig them back out of its own session, it fumbled around, because the edits spanned subagents and every restore came back partial. Then it quietly started rewriting them from memory, having decided in its panic that this was "easier".

• "It lost my design images after a compaction, then asked me to re-send them."

  The images were right there in the conversation. One compaction later the agent acted as if they had never existed. The compaction had handed it lossy text fragments in their place, and it believed them.

• "Which session does jazzy-twilight-sparkle.md belong to?"

  The plan file carries the architecture that matters. Its invented name says nothing about which session wrote it. The only record of that is a plan_mode attachment the session wrote when it entered Plan Mode, one line among many in its transcript.

• "The machine crashed with three sessions open. Which was which?"

  One was frontend, one backend, one debugging, and claude --resume gives you a lineup with no faces.

• "I literally just said that."

  A few compactions into a long run, the agent still knows its task. Everything you actually said and read is gone. Claude Code does not carry the last few turns across a compaction word for word, the way the coding agents Codex and pi do.

• "I DID tell you. In the question dialog."

  It searched its own session by user roles, found nothing, and concluded you never said it. You had said it in AskUserQuestion.

• "You're the orchestrator. Why can't you see which session is stuck?"

  Because, it explains, an unanswered AskUserQuestion is never written to disk at all, "so there's nothing I can watch." Or worse, it offers to "interpret it from the thinking."

• "You said it stopped, but it's waiting for a subagent!"

  It apologized, searched every other jsonl file and came back with a "root cause". Asked to orchestrate another session in the same project, it could not even tell whether that session had stopped. Another time it took a turn-duration record for the stop, though that record only times a single turn.

Each of these ends the same way. The agent says You are absolutely right., then stumbles through a one-off script over raw JSON and gets it subtly wrong.

✨ Highlights

1. ⏪ Recover files & deleted plans.

   csift recover replays every edit and read to restore a file's exact bytes at any point in time, as diff patches or as the final file.
   Shell traffic counts too. A heredoc or literal echo written through Bash, and a clean cat, head or sed -n read, replay as real content when the command's shape leaves no doubt about the bytes.
   Claude Code notes when a file changed outside its own edits, and csift stops at that point instead of guessing across it.
   Where such a change rules out an exact restore, it salvages what survived and marks the gaps.

2. 🖼 Images back out.

   csift image lists, dedups and extracts them by the same [Image #N] handle the session uses. (#N was never a unique id. csift copes.)
   A PDF read page by page comes back the same way, each image labelled with the page it actually is.

3. 📝 Plan ↔ Session, both directions.

   csift plan finds the plan a session wrote.
   csift plan --reverse ~/.claude/plans/jazzy-twilight-sparkle.md names the session that owns it.

4. 📇 Sessions you can tell apart.

   csift list names every session by its first and last messages, which the claude --resume picker never shows.
   A renamed or copied project directory can leave one session in two places.
   csift decides by comparing the two files' bytes, never their timestamps.
   A copy that matches the other or stops short of it merges into one, and copies that genuinely differ are both kept and both named.

5. 🧵 Un-clip a compaction.

   A compaction summary keeps the task state and drops most of the conversation.
   csift verbatim rebuilds the clipped back-and-forth within a --budget.
   Wired into a hook, it attaches the recent dialogue to every compaction.

   When Claude Code resumes a session, it reads back only as far as the latest compaction.
   --cross-compactions lets csift step back over a bounded number of them, so it can still tell which earlier turns the conversation kept and which it abandoned.
   Every command says how many it crossed and where it stopped.

6. 🏷 Typed search.

   A background task's completion notice is a "user" role.
   A subagent's return is also "user".
   Your AskUserQuestion answer is a tool result.
   csift has stepped in each of these traps already. csift search -t user.answer finds exactly what a naive grep swears was never said, and it is one of 73 {role}.{category}.{leaf} labels.

   Some labels name what the format hides.

   label	finds
   user.unsent	the message you esc-recalled and never actually sent
   user.queued	what you typed into the queue while a turn was running
   harness.turn.duration	the record behind the "Done in 1m 5s" line
   agent.thinking.narration	the API's one-line summaries, kept apart from the reasoning they summarize

7. 🛎 Pending questions, on the record.

   A hook csift ships records each AskUserQuestion, ExitPlanMode and MCP elicitation to a sidecar file.
   Every csift surface merges the unresolved ones in, transparently.
   An orchestrator can finally see which session is stuck waiting on a human, and on what.

8. 🩺 Is it actually stopped?

   A session can end its turn and still be running.

   • csift status gives one verdict and names its evidence (the session registry, the transcript tail, every child lane, the task list, a live process probe). A lane is one agent's thread of work, the main session or one of its subagents.
   • Every background shell, async agent and Monitor is listed with its age and whether it ever came back. Long sessions carry dozens of zombies.
   • csift wait --until stop --timeout 300 blocks until it really stops.
   • --ignore-background 'npm run dev' leaves out the tasks that never return by design.
   • On timeout it exits 124 and reports what happened while you waited.

   The timeout is required because some background tasks, a dev server for one, never return.
   Both commands read the live state at the moment they run, on macOS, Linux and Windows.

9. 📣 Message an agent Claude Code gives you no way to reach.

   Claude Code's own messaging tools reach some lanes and not others.
   An agent running inside a workflow script has no official channel.
   Neither does your own parent subagent, an idle session with no messaging socket open, or anything at all when you are outside Claude Code.

   • csift send @<lane> "stop after the current file" queues one message and says what will carry it.
   • Each send gets one verdict out of OK, FULL, MAY-FAIL, UNPREDICTABLE and REFUSED.
   • Delivery is a hook line you paste. csift deliver --recipe prints the block, and csift writes no settings file.
   • csift msg <id> compares what csift meant to deliver with what the receiver's own transcript shows arrived.
   • csift whoami --to @<lane> predicts reach and sends nothing.

   Where an official transport does exist, csift prints the exact call for you to make. It still queues its own copy, so the message survives even if that call is never made.

10. 🔎 Round-trips, not lines.

    A hit returns the whole exchange, rebuilt from the uuid / parentUuid graph.
    A matched tool call comes back with its result, and a user turn with the agent's reply.
    Neither grep nor Claude's ad-hoc scripts can reliably give you that context.

11. 🧭 What is this record even about?

    Every hit names the message it belongs to, and how many hops away that message sits.
    Where a record states its link to another one (a tool result to its call, a summary to the compaction it closes), the hit names that link with one of 27 fixed relation words.
    csift search "" @<uuid> --for L42 follows those links backward and returns every record that message led to, transitively.

12. 🌳 Subagent topology.

    The parent→child tree of every spawned agent, with each one's kind and lifecycle.
    Lanes frozen on a pending permission approval are detected.

13. 🤖 Designed for humans and LLMs.

    Output is terse, re-feedable and structured.
    csift ships a Claude Code skill, a set of instructions the agent loads when a task calls for it.
    With the skill installed, your Claude knows when and how to reach for csift.

14. 🔒 Local, plain, and it writes in two places only.

    Pure regex. No embeddings, no index, no database, no daemon, no network, no telemetry.
    Nothing is inferred without the output saying so.
    It reads files already on your disk and never touches your session histories.
    send, deliver and ack write into a csift-channel directory csift creates beside the session.
    recover, verbatim and image write only the path you name with an output flag. Nothing else is written.

15. ⚡ Rust + mmap + SIMD newline scan + byte prefilters + rayon.

    A search that matches nothing in a 397 MiB transcript runs in 0.63 to 0.73 s.
    Those are medians of 21 runs on an idle host, recorded in SPEC.md section 7h.
    Quick enough to call from inside a hook without noticing.

Install

cargo install csift

csift needs Rust 1.89+. To build from source instead:

git clone https://github.com/wdhwg001/csift.git
cd csift
cargo install --path .        # builds the optimized binary and puts `csift` on your PATH
# …or just `cargo build --release` → ./target/release/csift

By default csift reads ~/.claude. It reads another directory when given --claude-home <DIR> or Claude Code's own $CLAUDE_CONFIG_DIR. csift <command> --help is the full manual.

Teach it to your agent

The primary user of csift is the agent itself. Its output is terse and parseable, and every record carries a handle the agent can feed straight back in. The skill teaches Claude Code when and how to reach for it.

npx skills add wdhwg001/csift

From a clone, link the skill directory instead.

ln -s "$PWD/skills/csift" ~/.claude/skills/csift

Quickstart

Every command takes the shape csift <command> [TARGET] [flags]. A target is positional. It is @<uuid> for a session, @<agent-hex> for a subagent, a project path, or . for the current directory. With no target, csift scans every project.

csift <command> --help lists the full flag set, with examples.

The seventeen subcommands

What a compaction summary leaves out

When Claude Code compacts, it regenerates a dense summary, kilobytes standing in for the whole history before it. The summary is a good one. It keeps the key findings (what changed, with file:line) and often your standing directives verbatim. It is rewritten at every compaction, and what it is built to carry forward is the task.

csift verbatim carries the conversation instead, the verbatim User↔Agent exchange. That is what you actually said and what the agent actually reported back when it finished, with the hundreds of tool calls in between collapsed to a count. For the recent window it comes to tens of KB of full-fidelity dialogue where the summary kept a line or two. Your directive is in it, and so is the agent's "validated against HEAD, the premise holds, here's the evidence" report-back, which a selection of task findings does not carry.

The summary stays where it is. verbatim extends the post-compaction context with the recent conversation at full fidelity, budget-bounded and newest-first. It does not reach the oldest turns, and your never-compacted Plan file still owns the plan. The skill ships a SessionStart(compact) hook for this, compact-slice.sh, with its settings fragment in hooks.md. With the hook wired in, every compaction arrives with the recent verbatim conversation attached.

How it works

csift avoids loading a whole transcript wherever it can. It mmaps each .jsonl, scans newlines with SIMD (memchr), runs a cheap byte or regex prefilter, and parses only the candidate lines with serde_json. list reads just the head and tail of each file, and rayon fans the work out across files.

Most of the work is in reading Claude Code's log correctly. Most role:"user" records carry tool results, and only some are human turns. A pending question can stay off disk until it is answered. Compaction writes a specific shape. Subagents sit flat on disk, and their topology is rebuilt from the tool call that spawned each one. SPEC.md documents all of it, with the empirical grounding.

How much of this is verified

csift rests on hundreds of small facts about what Claude Code writes. Each one is a claim in INTROSPECTION.json. A claim carries the Claude Code version range it holds over and how completely it is attributed. Attribution asks whether the code that writes the fact was traced in the shipped Claude Code binary, and whether a specimen of the fact was observed. A claim keeps its current reading and at most one superseded one, and the table below counts the current reading of each.

Every range carries one receipt, and a receipt has to have measured something outside csift's own tree, such as a build, a byte offset in one, a corpus of transcripts or a live trial. The table is regenerated from the ledger, and the pre-commit gate checks it.

A traced claim pins each hop at a byte offset in that build, and the gate re-reads every one of them:

anchors byte-exact at Claude Code 2.1.258: 7,317 of 7,717 (elided 69, absent 30, prefix-only 80, read in another build 221), closed-state anchors 0, not checked

The five counts partition the total, and the gate asserts that they do. BYTE-EXACT is read over two encodings: a few excerpts were captured from regions that are not valid UTF-8 and stored one code point per byte, and 14 of these anchors match only under that second reading. An ELIDED excerpt carries an ellipsis somewhere, which is not the same as having no byte-exact form: 1,481 such excerpts ARE byte-exact at their own offsets and are counted with the rest, because minified JavaScript's spread operator wears the same three dots. What is left in the elided count really did elide something, and the gate holds each of those to the run BEFORE its first ellipsis at its own offset. An absent or prefix-only anchor is marked as such in the ledger, with the residue line that says how to find its producer again; those are the first thing the next pass re-locates. A hop read in an artifact the badge does not name carries that build and is compared against THAT one, which is why it is disclosed as a count and never as a verdict: whether a given machine holds those artifacts is a property of the machine. A closed state's anchors were exact at the build its own range ended on, so the gate counts them and does not re-read them against a later one.

A claim that is not end-to-end names the exact instrument that would close it. A claim attributed by elimination can never carry holds, the ledger's verdict for a claim re-observed exactly as written.

The project keeps private the method it used to take Claude Code apart. It does not publish a mechanical diff of the ledger either, since such a diff would expose the same method. It will not publish a corpus of transcripts. No level of redaction means anything for this format, because a redacted real transcript cannot be told apart from a hand-made one. A released corpus would prove nothing and could still leak.

Claude Code is conservative with its own jsonl, and the format rarely breaks across versions. So the ledger is re-verified periodically, and not in full for every Claude Code release. The sessions csift is developed in always run the current Claude Code, so a format change shows up in daily use between re-verifications.

Documentation

csift <command> --help is the manual written for people, and it carries the same information as the skill. Claude Code writes and maintains this whole repository, and its other documents are written for the agent that does that work.

• skills/csift/ is the skill an agent loads to operate csift. A short SKILL.md routes, one reference file covers each topic, and three hook scripts sit beside them.
• SPEC.md holds the design intent, the record-model semantics and the measurements behind the deep features.
• AGENTS.md is the repository's operating manual, covering the architecture, the jsonl domain knowledge and the quality gate.

They are dense and cross-referenced, and they are written for a model's attention. A person who wants to understand the codebase does best to hand the repository to their own agent and ask for the tour. It reads these files well.

Contributing

Claude wrote csift, so its bugs are Claude's own, and I don't think fixing them should be a human's job. The intended path is to have Claude clone the repo, clean up after itself and open the PR. From the PR onward, though, I expect a human who has read the code. If reviewing your PR would mean chatting with an AI, it should have been an Issue instead. Issues are genuinely welcome, and a clearly described problem is all I need to point my Claude at it.

The whole quality gate is one pre-commit hook, installed on your first cargo test. It runs the structure gate (file and folder limits), then cargo fmt --check, cargo clippy --all-targets -- -D warnings, the introspection ledger gate and cargo test. A failure at any stage blocks the commit.

License

Released under the MIT license.