feat(wrap): make tokensave the primary coding-task compressor, Serena the backup#1230
Conversation
… the backup `headroom wrap claude|codex` now registers tokensave (github.com/aovestdipaperino/tokensave) as the primary coding-task compressor — a local semantic code-graph MCP server (`tokensave serve`) the agent queries for symbols, call chains, and impact analysis instead of reading whole files. Serena becomes the backup: registered automatically only when tokensave is unavailable, or forced on with `--serena`. - Add headroom/graph/tokensave_installer.py: fetch the prebuilt release binary for the platform (release-binary only; no cargo), honoring HEADROOM_BINARIES_OFFLINE. Returns None on unsupported platform / failure so the caller falls back to Serena. - Add build_tokensave_spec() to the MCP registry; register/disable/migrate via the existing ServerSpec + ledger flow, mirroring Serena. - New flags: --no-tokensave (skip primary), --serena (force backup on); --no-serena now means "never register the backup". Default wrap removes a previously Headroom-installed Serena entry once tokensave is primary (user-managed entries are preserved). - Point --code-graph at tokensave; drop the codebase-memory-mcp install path (unwrap still cleans up any legacy codebase-memory-mcp entry). - unwrap claude/codex remove a Headroom-installed tokensave entry (ledger-gated). - Strands HeadroomBundle: tokensave primary (enable_tokensave_mcp=True), Serena backup (enable_serena_mcp now defaults False). - Tests: tokensave installer, register/disable/migrate, and the primary/backup policy; scoped offline guard keeps the CLI suite hermetic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR governanceThis PR follows the template and is marked ready for human review. |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Raises patch coverage on the tokensave integration: installer download failures / missing-binary archives / version-probe fallbacks (84% -> 99%), and the _ensure_tokensave_binary + _index_tokensave_project helpers (init/sync selection, non-zero, timeout, and missing-binary paths). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JerrettDavis
left a comment
There was a problem hiding this comment.
The direction may be worth exploring, but this is not ready as a default enterprise behavior. headroom wrap now downloads and executes a new third-party tokensave release binary by default, without checksum/signature verification or a pinned asset digest. That materially changes the trust boundary for every wrap user and makes CI/customer installs depend on mutable network release assets. Please keep tokensave opt-in until we have an integrity story (pinned checksums or bundled/reproducible artifact verification), clear offline behavior, and documentation that explains the new binary trust model. The Serena fallback/migration logic can be reviewed after the default-download risk is removed.
Addresses review on headroomlabs-ai#1230: headroom wrap downloads and executes the tokensave binary by default, so verify integrity before running it. - Pin a SHA-256 digest per release asset (TOKENSAVE_ASSET_DIGESTS) and verify downloaded bytes before extraction; a mismatch aborts the install (-> Serena fallback) instead of running unverified code. - Refuse assets with no pinned digest (e.g. a HEADROOM_TOKENSAVE_VERSION override) unless HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED=1 is set. - Bump pinned release v6.4.4 -> v7.0.0 with refreshed digests. - Document the binary trust model + offline behavior in proxy.mdx. - Tests for verify pass/mismatch/unpinned paths and abort-on-mismatch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks for the review @JerrettDavis — addressed the supply-chain concern in 57dcb96 so the default-download risk is removed:
Tests cover verify pass / digest mismatch / unpinned-refused / unpinned-with-optout, plus an end-to-end abort-on-mismatch asserting no binary is written. Full CLI + graph suites green (441 passed), ruff + mypy clean. The Serena fallback/migration logic is unchanged and ready whenever you'd like to take a look at that part. |
|
Note for follow-up (out of scope here): the same release-binary download pattern exists in Left untouched to keep this PR focused on the new tokensave default. Tracking a follow-up to lift the pinned-digest verification added here into a shared helper and apply it to rtk/lean_ctx/cbm. |
…y-compressor # Conflicts: # headroom/cli/wrap.py
Bump TOKENSAVE_VERSION v7.0.0 -> v7.0.2 and refresh the pinned SHA-256 digests for all release assets. tokensave v7.0.2 publishes a Windows aarch64 build, so add its digest and select x86_64 vs aarch64 in _detect_asset for Windows (matching the existing Linux logic). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JerrettDavis
left a comment
There was a problem hiding this comment.
Thanks for the follow-up here. The focused Tokensave coverage is in good shape locally: \ ests/test_graph_tokensave.py\, \ ests/test_cli/test_tokensave_setup.py\, and \ ests/test_cli/test_tokensave_helpers.py\ pass as 47/47.\n\nI do need to keep this in changes-requested for now because GitHub reports the branch as \DIRTY\, so this cannot be merged until the conflicts with current \main\ are resolved. I did not find a new code-level blocker in the focused slice I ran, but please update the branch and re-run the affected tests after resolving the conflicts so the final diff is reviewable against current main.
…ary-compressor # Conflicts: # headroom/cli/wrap.py
|
Rebased/merged Verified locally: tokensave + CLI suites pass ( |
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JerrettDavis
left a comment
There was a problem hiding this comment.
Re-reviewed after the merge/update commits. The previous default-download trust blocker has been addressed with pinned release digests and the branch is no longer dirty. The focused Tokensave/registry slice is green locally on the updated head.
Local validation run:
uv run --with pytest --with pytest-asyncio python -m pytest tests/test_graph_tokensave.py tests/test_cli/test_tokensave_setup.py tests/test_cli/test_tokensave_helpers.py tests/test_mcp_registry/test_install.py -q(58 passed)uv run ruff check headroom/graph/tokensave_installer.py headroom/mcp_registry/install.py headroom/cli/wrap.py tests/test_graph_tokensave.py tests/test_cli/test_tokensave_setup.py tests/test_cli/test_tokensave_helpers.py tests/test_mcp_registry/test_install.py
Approving the focused PR diff. GitHub currently shows only PR Governance checks on the latest head, not a full CI rollup.
Appreciate the callout! Can you please open an issue calling attention to these discrepancies for additional future hardening passes? |
|
Opened #1407 to track lifting the pinned-digest verification into a shared helper and applying it to |
🤖 I have created a release *beep* *boop* --- <details><summary>0.28.0</summary> ## [0.28.0](v0.27.0...v0.28.0) (2026-06-29) ### Features * add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback ([#1185](#1185)) ([f309244](f309244)) * add first-class OpenCode support (wrap, learn, mcp install) ([#559](#559)) ([91cd210](91cd210)) * add HEADROOM_KEEPALIVE_EXPIRY to keep upstream connections warm ([#1124](#1124)) ([85786b3](85786b3)) * **azure-foundry:** derive upstream URL from ANTHROPIC_FOUNDRY_RESOURCE ([#1138](#1138)) ([e5031b0](e5031b0)) * **cache:** attribute prompt-cache misses to TTL lapse vs prefix change ([#1313](#1313)) ([#1343](#1343)) ([4658721](4658721)) * **code:** add Perl support to code-aware compressor ([#1125](#1125)) ([f39858c](f39858c)) * headroom wrap opencode / unwrap opencode CLI ([#1105](#1105)) ([b4571cc](b4571cc)) * **learn:** weight loops in Headroom Learn + RTK-loop eval ([#1160](#1160)) ([14e8dc4](14e8dc4)) * **learn:** write per-project learnings to CLAUDE.local.md by default ([#1115](#1115)) ([ced75e4](ced75e4)) * **proxy:** add request timeout config ([#738](#738)) ([c0745d4](c0745d4)) * **proxy:** pilot hardening — inbound auth, security headers, audit log, air-gap switch ([#1537](#1537)) ([546ab55](546ab55)) * **proxy:** support glob patterns in exclude_tools ([#870](#870)) ([#1259](#1259)) ([a2159c0](a2159c0)) * **read-maturation:** activity-based hold-back Read maturation (Mechanism B) ([#1068](#1068)) ([723b80c](723b80c)) * **savings:** durable savings ledger + headroom savings command ([#1127](#1127)) ([978ffa0](978ffa0)) * **wrap:** add --1m to preserve the 1M context window on wrap claude ([#1158](#1158)) ([#1351](#1351)) ([b50d9c1](b50d9c1)) * **wrap:** make tokensave the primary coding-task compressor, Serena the backup ([#1230](#1230)) ([dca9853](dca9853)) ### Bug Fixes * **agent-evals:** Phase 0 — coding-agent accuracy A/B framework ([#1037](#1037)) ([84f9871](84f9871)) * **agno:** tolerate streaming tool-call SDK objects in parser ([#1312](#1312)) ([#1336](#1336)) ([5986c22](5986c22)) * **bedrock:** add boto3 1.41 + CRT for aws login credentials ([#1486](#1486)) ([4db3bc9](4db3bc9)) * bump codebase-memory-mcp to v0.8.1 ([#1284](#1284)) ([530318b](530318b)) * **ccr:** make headroom_retrieve a hash-only full-content lookup ([#1532](#1532)) ([c2fc4d3](c2fc4d3)) * **ccr:** propagate --no-ccr-marker flag to all compressors ([#1022](#1022)) ([#1197](#1197)) ([0c9b42a](0c9b42a)) * **ccr:** skip Anthropic marker emission when tool injection is deferred ([#1273](#1273)) ([2cae13d](2cae13d)) * **ci:** extend gitleaks allowlist to cover test fixtures + verified examples ([#1539](#1539)) ([d2565a6](d2565a6)) * **ci:** guarantee model present in test shards to end cache-miss flakiness ([#1399](#1399)) ([2e29c72](2e29c72)) * **ci:** normalize Windows CRLF line endings in PR governance script ([#1012](#1012)) ([5194388](5194388)) * **cli:** add explicit UTF-8 encoding to file I/O in wrap commands ([#1126](#1126)) ([#1164](#1164)) ([a0cb798](a0cb798)) * **cli:** fall back gracefully when embedding-server sidecar is absent ([#1206](#1206)) ([38f1404](38f1404)) * **cli:** harden all CLI surfaces + fix docs accuracy ([#1491](#1491)) ([bd76235](bd76235)) * **cli:** wire --http2/--no-http2 (HEADROOM_HTTP2) into proxy command ([#1373](#1373)) ([e06b616](e06b616)) * **cli:** wire --rpm/--tpm and HEADROOM_RPM/HEADROOM_TPM to the Click proxy command ([#1375](#1375)) ([8aab8f2](8aab8f2)) * **code:** slice tree-sitter byte offsets as UTF-8 ([#1332](#1332)) ([8238402](8238402)) * **code:** validate Python compressed syntax ([#1302](#1302)) ([cbd361d](cbd361d)) * **code:** verify a real parse in tree-sitter availability check ([#1231](#1231)) ([#1299](#1299)) ([5e0bb69](5e0bb69)) * **codex:** retag threads on init so Codex Desktop history stays visible ([#961](#961)) ([#1349](#1349)) ([e6bbc40](e6bbc40)) * **codex:** stop pinning Codex memory MCP to one project db ([#1269](#1269)) ([ad7993b](ad7993b)) * **dashboard:** include RTK stats in the historical tab ([#1324](#1324)) ([35939c3](35939c3)) * **deps:** remediate dependency CVEs and publish SBOM ([#1509](#1509)) ([5771a80](5771a80)) * **docker:** persist session history across container revisions ([#1118](#1118)) ([5912d65](5912d65)) * **gemini:** offload compression to the executor ([#1382](#1382)) ([615848e](615848e)) * **gemini:** resolve Google model capabilities through ModelRegistry ([#1276](#1276)) ([17ecad9](17ecad9)) * **install:** guard install_agent_ensure against duplicate runtime spawns ([#1301](#1301)) ([8da0b4e](8da0b4e)) * **install:** repair macOS launchd restart/start lifecycle ([#1290](#1290)) ([da1a397](da1a397)) * **install:** stop duplicating ENTRYPOINT in persistent-docker runtime command ([#833](#833)) ([#1348](#1348)) ([feedead](feedead)) * **io:** use UTF-8 with locale fallback and preserve line endings on config/text I/O ([#1498](#1498)) ([1baa04e](1baa04e)) * **kompress:** hard override keeps must-keep tokens regardless of model score ([#1400](#1400)) ([42612c8](42612c8)) * **langchain:** disable streaming on wrapped model during ainvoke() ([#1287](#1287)) ([3590046](3590046)) * **mcp:** register managed installs with a resolvable headroom command ([#1386](#1386)) ([22def93](22def93)) * **mcp:** report correct savings_percent in headroom_compress ([#1106](#1106)) ([f216e43](f216e43)) * **opencode:** write local MCP config ([#1381](#1381)) ([6c83790](6c83790)) * **packaging:** move hnswlib to optional [vector] extra so [all] needs no C++ toolchain ([#1499](#1499)) ([80fa086](80fa086)) * patch rtk hook script to use absolute path after register_claude_hooks ([#571](#571)) ([b618d2d](b618d2d)) * **perf:** surface RTK/CLI context-tool savings in perf and the session card ([#1433](#1433)) ([9362747](9362747)) * **proxy:** add --protect-tool-results to prevent lossy compression of exact-output Bash results ([#1374](#1374)) ([51d4bcf](51d4bcf)) * **proxy:** add an Anthropic buffered read-timeout override ([#1331](#1331)) ([3be2526](3be2526)) * **proxy:** add versionless Vertex AI routes for Claude Code compatibility ([#1321](#1321)) ([bb3e040](bb3e040)) * **proxy:** bind before eager preload so a hung compressor load can't block startup ([#1500](#1500)) ([d5ac07f](d5ac07f)) * **proxy:** build SSL contexts for custom CA bundles ([#1134](#1134)) ([561ba17](561ba17)) * **proxy:** forward request-id headers on the streaming path ([#1100](#1100)) ([#1258](#1258)) ([3d59df7](3d59df7)) * **proxy:** gate CCR retrieve/compress endpoints to loopback ([#1338](#1338)) ([acafb2d](acafb2d)) * **proxy:** honor force_kompress routing profile ([#996](#996)) ([b4682d6](b4682d6)) * **proxy:** keep large compression results on the critical path ([#296](#296)) ([#1352](#1352)) ([90734b6](90734b6)) * **proxy:** offload /v1/compress to the compression executor to stop blocking the loop ([#1501](#1501)) ([27e010e](27e010e)) * **proxy:** preserve Responses memory continuations with store=false ([#1103](#1103)) ([cdfeeac](cdfeeac)) * **proxy:** queue mid-turn user messages on non-Bedrock streaming path ([#1377](#1377)) ([b09f027](b09f027)) * **proxy:** register interceptor in explicit transforms list when HEADROOM_INTERCEPT_ENABLED ([#1376](#1376)) ([55c700c](55c700c)) * **proxy:** report real input tokens on streaming message_start ([#1132](#1132)) ([#1305](#1305)) ([70cc96a](70cc96a)) * **proxy:** retry upstream 429 with Retry-After on both forwarders ([#1329](#1329)) ([90bee89](90bee89)) * **proxy:** retry upstream 529 overloaded like 429 on both forwarders ([#1495](#1495)) ([547b15d](547b15d)) * **proxy:** stop re-compressing headroom_retrieve output and emitting unredeemable markers ([#1323](#1323)) ([43494ff](43494ff)) * **proxy:** strip Codex lite header from OpenAI WebSockets ([#1543](#1543)) ([5d3803a](5d3803a)) * **read-lifecycle:** persist STALE Read originals in the CCR store ([#1488](#1488)) ([9157173](9157173)) * recover persistent proxy feature checks and reject non-Copilot exchange URL ([#1465](#1465)) ([16c638b](16c638b)) * remove agents.md ([#1540](#1540)) ([a7d3360](a7d3360)) * respect COPILOT_PROVIDER_TYPE env var when provider_type is auto ([#549](#549)) ([24cf256](24cf256)) * restore token-mode compression on frozen prefixes ([#1489](#1489)) ([8e0dadf](8e0dadf)) * **router:** degrade to pure-Python detection on native panic ([#1123](#1123)) ([#1260](#1260)) ([a00fb67](a00fb67)) * **rtk:** stop hook registration timing out on a forked daemon ([#1314](#1314)) ([9758817](9758817)) * **smart-crusher:** honor enable_ccr_marker on the opaque-blob path ([#1130](#1130)) ([27d6f8e](27d6f8e)) * **subscription:** only reset 5h contribution on real rollover, not API jitter ([#1255](#1255)) ([8d6c175](8d6c175)) * **subscription:** run transcript token scan off the event loop ([#1263](#1263)) ([f03021f](f03021f)) * surface output reduction without a restart, and explain $0.00 savings on Python 3.14 ([#1296](#1296)) ([c30ec4c](c30ec4c)) * **tests:** reset whole headroom logger subtree so caplog stays deterministic ([#1117](#1117)) ([fda4670](fda4670)) * **tls:** add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection ([#1308](#1308)) ([#1341](#1341)) ([52068dd](52068dd)) * **tokenizers:** price CJK/Kana/Hangul at ~1 token per char in EstimatingTokenCounter ([#1093](#1093)) ([a35fe86](a35fe86)) * **transforms:** gate tool string output from lossy compression ([#1307](#1307)) ([#1387](#1387)) ([c6c921a](c6c921a)) * **websocket:** harden responses websocket origin handling ([#1481](#1481)) ([c632023](c632023)) * **windows:** pin UTF-8 encoding on text-mode subprocess calls ([#1311](#1311)) ([d633e81](d633e81)) * **wrap:** add Copilot unwrap command ([#1251](#1251)) ([b4fde0c](b4fde0c)) * **wrap:** isolate proxy stdio from proxy.log on Windows ([#1191](#1191)) ([959ab0d](959ab0d)) * **wrap:** keep agent savings opt-in ([#1294](#1294)) ([b829ceb](b829ceb)) * **wrap:** show the dashboard URL when the proxy is already running ([#1313](#1313)) ([b0146c4](b0146c4)) ### Performance Improvements * **compression:** take large cold-start contexts off the synchronous kompress path ([#1171](#1171)) ([#1298](#1298)) ([6c68ff4](6c68ff4)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
… the backup (headroomlabs-ai#1230) ## Description Makes **tokensave** ([github.com/aovestdipaperino/tokensave](https://github.com/aovestdipaperino/tokensave)) the **primary coding-task compressor** that `headroom wrap` installs, and demotes **Serena** to a **backup**. tokensave is a local semantic code-graph MCP server (`tokensave serve`): the agent queries it for symbols, call chains, and impact analysis instead of grepping/reading whole files — the same role Serena filled, but as a pre-indexed graph. Serena now only registers when tokensave is unavailable (or when forced with `--serena`). Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/graph/tokensave_installer.py` (new): fetch the prebuilt tokensave release binary for the platform (release-binary only — no `cargo` compile at wrap time); honors `HEADROOM_BINARIES_OFFLINE`; returns `None` (→ Serena) when no asset exists (e.g. x86_64 macOS) or the download fails. - `mcp_registry`: `build_tokensave_spec()`; registration/disable/migrate go through the existing `ServerSpec` + ownership-ledger flow, identical to Serena. - `cli/wrap.py`: new `_setup_coding_compressor` primary/backup policy; tokensave setup/disable/migrate/index helpers. New flags `--no-tokensave` (skip primary) and `--serena` (force backup on); `--no-serena` now means "never register the backup". Default wrap removes a previously Headroom-installed Serena entry once tokensave is primary (user-managed entries preserved). `--code-graph` repointed to tokensave; the legacy `codebase-memory-mcp` install path is dropped (unwrap still cleans up legacy entries). `unwrap claude|codex` remove a ledger-owned tokensave entry. - Strands `HeadroomBundle`: `enable_tokensave_mcp=True` (primary); `enable_serena_mcp` now defaults `False` (backup). - `docs/content/docs/proxy.mdx`: `--code-graph` description updated from codebase-memory-mcp to tokensave. - Tests: tokensave installer (incl. error paths), register/disable/migrate, primary/backup policy, and the binary-resolution/indexing helpers. A scoped `tests/test_cli/conftest.py` offline guard keeps the CLI suite hermetic. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run pytest -q tests/test_graph_tokensave.py tests/test_cli/test_tokensave_setup.py tests/test_cli/test_tokensave_helpers.py 41 passed $ uv run pytest -q tests/test_cli/ tests/test_graph.py tests/test_graph_tokensave.py 421 passed # full CLI + graph suites, incl. all pre-existing Serena/unwrap/registry tests $ uv run pytest -q tests/test_mcp_registry/ tests/test_proxy_healthchecks.py passed $ uv run ruff format --check headroom/ tests/ # 822 files already formatted $ uv run ruff check <changed files> # All checks passed! $ uv run mypy headroom/graph/tokensave_installer.py headroom/mcp_registry/install.py Success: no issues found in 2 source files # Coverage on new module headroom/graph/tokensave_installer.py 99% ``` ## Real Behavior Proof - Environment: macOS (darwin arm64), Python 3.14, `uv` dev env; tokensave 7.0.2 binary present on PATH and exercised against this repo's `.tokensave/` graph during development. The installer pins release **v7.0.2** (SHA-256-verified) across macOS arm64, Linux aarch64/x86_64, and Windows x86_64/aarch64. - Exact command / steps: `headroom wrap claude` registers `tokensave serve` as the primary MCP code-graph server and indexes the project; with the binary removed from PATH and `HEADROOM_BINARIES_OFFLINE=1`, the same command falls back to registering Serena. Behavior is pinned by the unit tests (binary-present → tokensave registered + Serena entry removed; binary-absent → Serena fallback; `--serena` forces backup on; `--no-serena` suppresses it; `--no-tokensave` disables primary). - Observed result: tokensave registered as primary on the binary-present path; Serena registered on the unavailable path; unwrap removes only ledger-owned entries. - Not tested: live end-to-end agent session inside Claude Code / Codex against a real provider API; Windows/Linux release-asset download (covered by unit tests with mocked archives, not a live fetch). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG is left untouched: this repo generates it via release-please from Conventional Commits, so a manual edit is N/A. - `strands/bundle.py` shows 0% patch coverage because that module hard-imports the optional `strands` SDK, which CI does not install (the pre-existing `_make_serena_client` was likewise uncovered) — not a regression. - A `test (3)` shard failure on `headroom.memory.bridge` is a pre-existing offline-CI flake (cannot reach huggingface.co); it touches no file in this PR and the scoped offline guard only applies under `tests/test_cli/`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
🤖 I have created a release *beep* *boop* --- <details><summary>0.28.0</summary> ## [0.28.0](headroomlabs-ai/headroom@v0.27.0...v0.28.0) (2026-06-29) ### Features * add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback ([headroomlabs-ai#1185](headroomlabs-ai#1185)) ([f309244](headroomlabs-ai@f309244)) * add first-class OpenCode support (wrap, learn, mcp install) ([headroomlabs-ai#559](headroomlabs-ai#559)) ([91cd210](headroomlabs-ai@91cd210)) * add HEADROOM_KEEPALIVE_EXPIRY to keep upstream connections warm ([headroomlabs-ai#1124](headroomlabs-ai#1124)) ([85786b3](headroomlabs-ai@85786b3)) * **azure-foundry:** derive upstream URL from ANTHROPIC_FOUNDRY_RESOURCE ([headroomlabs-ai#1138](headroomlabs-ai#1138)) ([e5031b0](headroomlabs-ai@e5031b0)) * **cache:** attribute prompt-cache misses to TTL lapse vs prefix change ([headroomlabs-ai#1313](headroomlabs-ai#1313)) ([headroomlabs-ai#1343](headroomlabs-ai#1343)) ([4658721](headroomlabs-ai@4658721)) * **code:** add Perl support to code-aware compressor ([headroomlabs-ai#1125](headroomlabs-ai#1125)) ([f39858c](headroomlabs-ai@f39858c)) * headroom wrap opencode / unwrap opencode CLI ([headroomlabs-ai#1105](headroomlabs-ai#1105)) ([b4571cc](headroomlabs-ai@b4571cc)) * **learn:** weight loops in Headroom Learn + RTK-loop eval ([headroomlabs-ai#1160](headroomlabs-ai#1160)) ([14e8dc4](headroomlabs-ai@14e8dc4)) * **learn:** write per-project learnings to CLAUDE.local.md by default ([headroomlabs-ai#1115](headroomlabs-ai#1115)) ([ced75e4](headroomlabs-ai@ced75e4)) * **proxy:** add request timeout config ([headroomlabs-ai#738](headroomlabs-ai#738)) ([c0745d4](headroomlabs-ai@c0745d4)) * **proxy:** pilot hardening — inbound auth, security headers, audit log, air-gap switch ([headroomlabs-ai#1537](headroomlabs-ai#1537)) ([546ab55](headroomlabs-ai@546ab55)) * **proxy:** support glob patterns in exclude_tools ([headroomlabs-ai#870](headroomlabs-ai#870)) ([headroomlabs-ai#1259](headroomlabs-ai#1259)) ([a2159c0](headroomlabs-ai@a2159c0)) * **read-maturation:** activity-based hold-back Read maturation (Mechanism B) ([headroomlabs-ai#1068](headroomlabs-ai#1068)) ([723b80c](headroomlabs-ai@723b80c)) * **savings:** durable savings ledger + headroom savings command ([headroomlabs-ai#1127](headroomlabs-ai#1127)) ([978ffa0](headroomlabs-ai@978ffa0)) * **wrap:** add --1m to preserve the 1M context window on wrap claude ([headroomlabs-ai#1158](headroomlabs-ai#1158)) ([headroomlabs-ai#1351](headroomlabs-ai#1351)) ([b50d9c1](headroomlabs-ai@b50d9c1)) * **wrap:** make tokensave the primary coding-task compressor, Serena the backup ([headroomlabs-ai#1230](headroomlabs-ai#1230)) ([dca9853](headroomlabs-ai@dca9853)) ### Bug Fixes * **agent-evals:** Phase 0 — coding-agent accuracy A/B framework ([headroomlabs-ai#1037](headroomlabs-ai#1037)) ([84f9871](headroomlabs-ai@84f9871)) * **agno:** tolerate streaming tool-call SDK objects in parser ([headroomlabs-ai#1312](headroomlabs-ai#1312)) ([headroomlabs-ai#1336](headroomlabs-ai#1336)) ([5986c22](headroomlabs-ai@5986c22)) * **bedrock:** add boto3 1.41 + CRT for aws login credentials ([headroomlabs-ai#1486](headroomlabs-ai#1486)) ([4db3bc9](headroomlabs-ai@4db3bc9)) * bump codebase-memory-mcp to v0.8.1 ([headroomlabs-ai#1284](headroomlabs-ai#1284)) ([530318b](headroomlabs-ai@530318b)) * **ccr:** make headroom_retrieve a hash-only full-content lookup ([headroomlabs-ai#1532](headroomlabs-ai#1532)) ([c2fc4d3](headroomlabs-ai@c2fc4d3)) * **ccr:** propagate --no-ccr-marker flag to all compressors ([headroomlabs-ai#1022](headroomlabs-ai#1022)) ([headroomlabs-ai#1197](headroomlabs-ai#1197)) ([0c9b42a](headroomlabs-ai@0c9b42a)) * **ccr:** skip Anthropic marker emission when tool injection is deferred ([headroomlabs-ai#1273](headroomlabs-ai#1273)) ([2cae13d](headroomlabs-ai@2cae13d)) * **ci:** extend gitleaks allowlist to cover test fixtures + verified examples ([headroomlabs-ai#1539](headroomlabs-ai#1539)) ([d2565a6](headroomlabs-ai@d2565a6)) * **ci:** guarantee model present in test shards to end cache-miss flakiness ([headroomlabs-ai#1399](headroomlabs-ai#1399)) ([2e29c72](headroomlabs-ai@2e29c72)) * **ci:** normalize Windows CRLF line endings in PR governance script ([headroomlabs-ai#1012](headroomlabs-ai#1012)) ([5194388](headroomlabs-ai@5194388)) * **cli:** add explicit UTF-8 encoding to file I/O in wrap commands ([headroomlabs-ai#1126](headroomlabs-ai#1126)) ([headroomlabs-ai#1164](headroomlabs-ai#1164)) ([a0cb798](headroomlabs-ai@a0cb798)) * **cli:** fall back gracefully when embedding-server sidecar is absent ([headroomlabs-ai#1206](headroomlabs-ai#1206)) ([38f1404](headroomlabs-ai@38f1404)) * **cli:** harden all CLI surfaces + fix docs accuracy ([headroomlabs-ai#1491](headroomlabs-ai#1491)) ([bd76235](headroomlabs-ai@bd76235)) * **cli:** wire --http2/--no-http2 (HEADROOM_HTTP2) into proxy command ([headroomlabs-ai#1373](headroomlabs-ai#1373)) ([e06b616](headroomlabs-ai@e06b616)) * **cli:** wire --rpm/--tpm and HEADROOM_RPM/HEADROOM_TPM to the Click proxy command ([headroomlabs-ai#1375](headroomlabs-ai#1375)) ([8aab8f2](headroomlabs-ai@8aab8f2)) * **code:** slice tree-sitter byte offsets as UTF-8 ([headroomlabs-ai#1332](headroomlabs-ai#1332)) ([8238402](headroomlabs-ai@8238402)) * **code:** validate Python compressed syntax ([headroomlabs-ai#1302](headroomlabs-ai#1302)) ([cbd361d](headroomlabs-ai@cbd361d)) * **code:** verify a real parse in tree-sitter availability check ([headroomlabs-ai#1231](headroomlabs-ai#1231)) ([headroomlabs-ai#1299](headroomlabs-ai#1299)) ([5e0bb69](headroomlabs-ai@5e0bb69)) * **codex:** retag threads on init so Codex Desktop history stays visible ([headroomlabs-ai#961](headroomlabs-ai#961)) ([headroomlabs-ai#1349](headroomlabs-ai#1349)) ([e6bbc40](headroomlabs-ai@e6bbc40)) * **codex:** stop pinning Codex memory MCP to one project db ([headroomlabs-ai#1269](headroomlabs-ai#1269)) ([ad7993b](headroomlabs-ai@ad7993b)) * **dashboard:** include RTK stats in the historical tab ([headroomlabs-ai#1324](headroomlabs-ai#1324)) ([35939c3](headroomlabs-ai@35939c3)) * **deps:** remediate dependency CVEs and publish SBOM ([headroomlabs-ai#1509](headroomlabs-ai#1509)) ([5771a80](headroomlabs-ai@5771a80)) * **docker:** persist session history across container revisions ([headroomlabs-ai#1118](headroomlabs-ai#1118)) ([5912d65](headroomlabs-ai@5912d65)) * **gemini:** offload compression to the executor ([headroomlabs-ai#1382](headroomlabs-ai#1382)) ([615848e](headroomlabs-ai@615848e)) * **gemini:** resolve Google model capabilities through ModelRegistry ([headroomlabs-ai#1276](headroomlabs-ai#1276)) ([17ecad9](headroomlabs-ai@17ecad9)) * **install:** guard install_agent_ensure against duplicate runtime spawns ([headroomlabs-ai#1301](headroomlabs-ai#1301)) ([8da0b4e](headroomlabs-ai@8da0b4e)) * **install:** repair macOS launchd restart/start lifecycle ([headroomlabs-ai#1290](headroomlabs-ai#1290)) ([da1a397](headroomlabs-ai@da1a397)) * **install:** stop duplicating ENTRYPOINT in persistent-docker runtime command ([headroomlabs-ai#833](headroomlabs-ai#833)) ([headroomlabs-ai#1348](headroomlabs-ai#1348)) ([feedead](headroomlabs-ai@feedead)) * **io:** use UTF-8 with locale fallback and preserve line endings on config/text I/O ([headroomlabs-ai#1498](headroomlabs-ai#1498)) ([1baa04e](headroomlabs-ai@1baa04e)) * **kompress:** hard override keeps must-keep tokens regardless of model score ([headroomlabs-ai#1400](headroomlabs-ai#1400)) ([42612c8](headroomlabs-ai@42612c8)) * **langchain:** disable streaming on wrapped model during ainvoke() ([headroomlabs-ai#1287](headroomlabs-ai#1287)) ([3590046](headroomlabs-ai@3590046)) * **mcp:** register managed installs with a resolvable headroom command ([headroomlabs-ai#1386](headroomlabs-ai#1386)) ([22def93](headroomlabs-ai@22def93)) * **mcp:** report correct savings_percent in headroom_compress ([headroomlabs-ai#1106](headroomlabs-ai#1106)) ([f216e43](headroomlabs-ai@f216e43)) * **opencode:** write local MCP config ([headroomlabs-ai#1381](headroomlabs-ai#1381)) ([6c83790](headroomlabs-ai@6c83790)) * **packaging:** move hnswlib to optional [vector] extra so [all] needs no C++ toolchain ([headroomlabs-ai#1499](headroomlabs-ai#1499)) ([80fa086](headroomlabs-ai@80fa086)) * patch rtk hook script to use absolute path after register_claude_hooks ([headroomlabs-ai#571](headroomlabs-ai#571)) ([b618d2d](headroomlabs-ai@b618d2d)) * **perf:** surface RTK/CLI context-tool savings in perf and the session card ([headroomlabs-ai#1433](headroomlabs-ai#1433)) ([9362747](headroomlabs-ai@9362747)) * **proxy:** add --protect-tool-results to prevent lossy compression of exact-output Bash results ([headroomlabs-ai#1374](headroomlabs-ai#1374)) ([51d4bcf](headroomlabs-ai@51d4bcf)) * **proxy:** add an Anthropic buffered read-timeout override ([headroomlabs-ai#1331](headroomlabs-ai#1331)) ([3be2526](headroomlabs-ai@3be2526)) * **proxy:** add versionless Vertex AI routes for Claude Code compatibility ([headroomlabs-ai#1321](headroomlabs-ai#1321)) ([bb3e040](headroomlabs-ai@bb3e040)) * **proxy:** bind before eager preload so a hung compressor load can't block startup ([headroomlabs-ai#1500](headroomlabs-ai#1500)) ([d5ac07f](headroomlabs-ai@d5ac07f)) * **proxy:** build SSL contexts for custom CA bundles ([headroomlabs-ai#1134](headroomlabs-ai#1134)) ([561ba17](headroomlabs-ai@561ba17)) * **proxy:** forward request-id headers on the streaming path ([headroomlabs-ai#1100](headroomlabs-ai#1100)) ([headroomlabs-ai#1258](headroomlabs-ai#1258)) ([3d59df7](headroomlabs-ai@3d59df7)) * **proxy:** gate CCR retrieve/compress endpoints to loopback ([headroomlabs-ai#1338](headroomlabs-ai#1338)) ([acafb2d](headroomlabs-ai@acafb2d)) * **proxy:** honor force_kompress routing profile ([headroomlabs-ai#996](headroomlabs-ai#996)) ([b4682d6](headroomlabs-ai@b4682d6)) * **proxy:** keep large compression results on the critical path ([headroomlabs-ai#296](headroomlabs-ai#296)) ([headroomlabs-ai#1352](headroomlabs-ai#1352)) ([90734b6](headroomlabs-ai@90734b6)) * **proxy:** offload /v1/compress to the compression executor to stop blocking the loop ([headroomlabs-ai#1501](headroomlabs-ai#1501)) ([27e010e](headroomlabs-ai@27e010e)) * **proxy:** preserve Responses memory continuations with store=false ([headroomlabs-ai#1103](headroomlabs-ai#1103)) ([cdfeeac](headroomlabs-ai@cdfeeac)) * **proxy:** queue mid-turn user messages on non-Bedrock streaming path ([headroomlabs-ai#1377](headroomlabs-ai#1377)) ([b09f027](headroomlabs-ai@b09f027)) * **proxy:** register interceptor in explicit transforms list when HEADROOM_INTERCEPT_ENABLED ([headroomlabs-ai#1376](headroomlabs-ai#1376)) ([55c700c](headroomlabs-ai@55c700c)) * **proxy:** report real input tokens on streaming message_start ([headroomlabs-ai#1132](headroomlabs-ai#1132)) ([headroomlabs-ai#1305](headroomlabs-ai#1305)) ([70cc96a](headroomlabs-ai@70cc96a)) * **proxy:** retry upstream 429 with Retry-After on both forwarders ([headroomlabs-ai#1329](headroomlabs-ai#1329)) ([90bee89](headroomlabs-ai@90bee89)) * **proxy:** retry upstream 529 overloaded like 429 on both forwarders ([headroomlabs-ai#1495](headroomlabs-ai#1495)) ([547b15d](headroomlabs-ai@547b15d)) * **proxy:** stop re-compressing headroom_retrieve output and emitting unredeemable markers ([headroomlabs-ai#1323](headroomlabs-ai#1323)) ([43494ff](headroomlabs-ai@43494ff)) * **proxy:** strip Codex lite header from OpenAI WebSockets ([headroomlabs-ai#1543](headroomlabs-ai#1543)) ([5d3803a](headroomlabs-ai@5d3803a)) * **read-lifecycle:** persist STALE Read originals in the CCR store ([headroomlabs-ai#1488](headroomlabs-ai#1488)) ([9157173](headroomlabs-ai@9157173)) * recover persistent proxy feature checks and reject non-Copilot exchange URL ([headroomlabs-ai#1465](headroomlabs-ai#1465)) ([16c638b](headroomlabs-ai@16c638b)) * remove agents.md ([headroomlabs-ai#1540](headroomlabs-ai#1540)) ([a7d3360](headroomlabs-ai@a7d3360)) * respect COPILOT_PROVIDER_TYPE env var when provider_type is auto ([headroomlabs-ai#549](headroomlabs-ai#549)) ([24cf256](headroomlabs-ai@24cf256)) * restore token-mode compression on frozen prefixes ([headroomlabs-ai#1489](headroomlabs-ai#1489)) ([8e0dadf](headroomlabs-ai@8e0dadf)) * **router:** degrade to pure-Python detection on native panic ([headroomlabs-ai#1123](headroomlabs-ai#1123)) ([headroomlabs-ai#1260](headroomlabs-ai#1260)) ([a00fb67](headroomlabs-ai@a00fb67)) * **rtk:** stop hook registration timing out on a forked daemon ([headroomlabs-ai#1314](headroomlabs-ai#1314)) ([9758817](headroomlabs-ai@9758817)) * **smart-crusher:** honor enable_ccr_marker on the opaque-blob path ([headroomlabs-ai#1130](headroomlabs-ai#1130)) ([27d6f8e](headroomlabs-ai@27d6f8e)) * **subscription:** only reset 5h contribution on real rollover, not API jitter ([headroomlabs-ai#1255](headroomlabs-ai#1255)) ([8d6c175](headroomlabs-ai@8d6c175)) * **subscription:** run transcript token scan off the event loop ([headroomlabs-ai#1263](headroomlabs-ai#1263)) ([f03021f](headroomlabs-ai@f03021f)) * surface output reduction without a restart, and explain $0.00 savings on Python 3.14 ([headroomlabs-ai#1296](headroomlabs-ai#1296)) ([c30ec4c](headroomlabs-ai@c30ec4c)) * **tests:** reset whole headroom logger subtree so caplog stays deterministic ([headroomlabs-ai#1117](headroomlabs-ai#1117)) ([fda4670](headroomlabs-ai@fda4670)) * **tls:** add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection ([headroomlabs-ai#1308](headroomlabs-ai#1308)) ([headroomlabs-ai#1341](headroomlabs-ai#1341)) ([52068dd](headroomlabs-ai@52068dd)) * **tokenizers:** price CJK/Kana/Hangul at ~1 token per char in EstimatingTokenCounter ([headroomlabs-ai#1093](headroomlabs-ai#1093)) ([a35fe86](headroomlabs-ai@a35fe86)) * **transforms:** gate tool string output from lossy compression ([headroomlabs-ai#1307](headroomlabs-ai#1307)) ([headroomlabs-ai#1387](headroomlabs-ai#1387)) ([c6c921a](headroomlabs-ai@c6c921a)) * **websocket:** harden responses websocket origin handling ([headroomlabs-ai#1481](headroomlabs-ai#1481)) ([c632023](headroomlabs-ai@c632023)) * **windows:** pin UTF-8 encoding on text-mode subprocess calls ([headroomlabs-ai#1311](headroomlabs-ai#1311)) ([d633e81](headroomlabs-ai@d633e81)) * **wrap:** add Copilot unwrap command ([headroomlabs-ai#1251](headroomlabs-ai#1251)) ([b4fde0c](headroomlabs-ai@b4fde0c)) * **wrap:** isolate proxy stdio from proxy.log on Windows ([headroomlabs-ai#1191](headroomlabs-ai#1191)) ([959ab0d](headroomlabs-ai@959ab0d)) * **wrap:** keep agent savings opt-in ([headroomlabs-ai#1294](headroomlabs-ai#1294)) ([b829ceb](headroomlabs-ai@b829ceb)) * **wrap:** show the dashboard URL when the proxy is already running ([headroomlabs-ai#1313](headroomlabs-ai#1313)) ([b0146c4](headroomlabs-ai@b0146c4)) ### Performance Improvements * **compression:** take large cold-start contexts off the synchronous kompress path ([headroomlabs-ai#1171](headroomlabs-ai#1171)) ([headroomlabs-ai#1298](headroomlabs-ai#1298)) ([6c68ff4](headroomlabs-ai@6c68ff4)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Description
Makes tokensave (github.com/aovestdipaperino/tokensave) the primary coding-task compressor that
headroom wrapinstalls, and demotes Serena to a backup. tokensave is a local semantic code-graph MCP server (tokensave serve): the agent queries it for symbols, call chains, and impact analysis instead of grepping/reading whole files — the same role Serena filled, but as a pre-indexed graph. Serena now only registers when tokensave is unavailable (or when forced with--serena).Closes #
Type of Change
Changes Made
headroom/graph/tokensave_installer.py(new): fetch the prebuilt tokensave release binary for the platform (release-binary only — nocargocompile at wrap time); honorsHEADROOM_BINARIES_OFFLINE; returnsNone(→ Serena) when no asset exists (e.g. x86_64 macOS) or the download fails.mcp_registry:build_tokensave_spec(); registration/disable/migrate go through the existingServerSpec+ ownership-ledger flow, identical to Serena.cli/wrap.py: new_setup_coding_compressorprimary/backup policy; tokensave setup/disable/migrate/index helpers. New flags--no-tokensave(skip primary) and--serena(force backup on);--no-serenanow means "never register the backup". Default wrap removes a previously Headroom-installed Serena entry once tokensave is primary (user-managed entries preserved).--code-graphrepointed to tokensave; the legacycodebase-memory-mcpinstall path is dropped (unwrap still cleans up legacy entries).unwrap claude|codexremove a ledger-owned tokensave entry.HeadroomBundle:enable_tokensave_mcp=True(primary);enable_serena_mcpnow defaultsFalse(backup).docs/content/docs/proxy.mdx:--code-graphdescription updated from codebase-memory-mcp to tokensave.tests/test_cli/conftest.pyoffline guard keeps the CLI suite hermetic.Testing
pytest)ruff check .)mypy headroom)Test Output
Real Behavior Proof
uvdev env; tokensave 7.0.2 binary present on PATH and exercised against this repo's.tokensave/graph during development. The installer pins release v7.0.2 (SHA-256-verified) across macOS arm64, Linux aarch64/x86_64, and Windows x86_64/aarch64.headroom wrap clauderegisterstokensave serveas the primary MCP code-graph server and indexes the project; with the binary removed from PATH andHEADROOM_BINARIES_OFFLINE=1, the same command falls back to registering Serena. Behavior is pinned by the unit tests (binary-present → tokensave registered + Serena entry removed; binary-absent → Serena fallback;--serenaforces backup on;--no-serenasuppresses it;--no-tokensavedisables primary).Review Readiness
Checklist
Additional Notes
strands/bundle.pyshows 0% patch coverage because that module hard-imports the optionalstrandsSDK, which CI does not install (the pre-existing_make_serena_clientwas likewise uncovered) — not a regression.test (3)shard failure onheadroom.memory.bridgeis a pre-existing offline-CI flake (cannot reach huggingface.co); it touches no file in this PR and the scoped offline guard only applies undertests/test_cli/.🤖 Generated with Claude Code