fix(perf): surface RTK/CLI context-tool savings in perf and the session card#1433
Merged
Conversation
…on card `headroom perf` read only proxy.log compression records, so RTK savings (its own lifetime counter, never logged to proxy.log) were invisible — perf reported "token savings" while silently dropping the entire CLI-filtering layer. The dashboard Session card likewise showed only the session-delta (~0 right after a proxy restart), unlabeled and with no lifetime figure. - perf: format_report/build_perf_summary now attach RTK lifetime savings (best-effort via _get_context_tool_stats().lifetime; omitted when no tool is installed), in both the text report and JSON output. - dashboard: Session card labels the RTK number 'this session', uses the real session_savings_pct, and shows lifetime alongside it. - tests: real _get_context_tool_stats() session-delta regression + perf RTK-surfacing tests.
Contributor
PR governanceThis PR follows the template and is marked ready for human review. |
Merged
chopratejas
pushed a commit
that referenced
this pull request
Jun 29, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>0.28.0</summary> ## [0.28.0](v0.27.0...v0.28.0) (2026-06-29) ### Features * add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback ([#1185](#1185)) ([f309244](f309244)) * add first-class OpenCode support (wrap, learn, mcp install) ([#559](#559)) ([91cd210](91cd210)) * add HEADROOM_KEEPALIVE_EXPIRY to keep upstream connections warm ([#1124](#1124)) ([85786b3](85786b3)) * **azure-foundry:** derive upstream URL from ANTHROPIC_FOUNDRY_RESOURCE ([#1138](#1138)) ([e5031b0](e5031b0)) * **cache:** attribute prompt-cache misses to TTL lapse vs prefix change ([#1313](#1313)) ([#1343](#1343)) ([4658721](4658721)) * **code:** add Perl support to code-aware compressor ([#1125](#1125)) ([f39858c](f39858c)) * headroom wrap opencode / unwrap opencode CLI ([#1105](#1105)) ([b4571cc](b4571cc)) * **learn:** weight loops in Headroom Learn + RTK-loop eval ([#1160](#1160)) ([14e8dc4](14e8dc4)) * **learn:** write per-project learnings to CLAUDE.local.md by default ([#1115](#1115)) ([ced75e4](ced75e4)) * **proxy:** add request timeout config ([#738](#738)) ([c0745d4](c0745d4)) * **proxy:** pilot hardening — inbound auth, security headers, audit log, air-gap switch ([#1537](#1537)) ([546ab55](546ab55)) * **proxy:** support glob patterns in exclude_tools ([#870](#870)) ([#1259](#1259)) ([a2159c0](a2159c0)) * **read-maturation:** activity-based hold-back Read maturation (Mechanism B) ([#1068](#1068)) ([723b80c](723b80c)) * **savings:** durable savings ledger + headroom savings command ([#1127](#1127)) ([978ffa0](978ffa0)) * **wrap:** add --1m to preserve the 1M context window on wrap claude ([#1158](#1158)) ([#1351](#1351)) ([b50d9c1](b50d9c1)) * **wrap:** make tokensave the primary coding-task compressor, Serena the backup ([#1230](#1230)) ([dca9853](dca9853)) ### Bug Fixes * **agent-evals:** Phase 0 — coding-agent accuracy A/B framework ([#1037](#1037)) ([84f9871](84f9871)) * **agno:** tolerate streaming tool-call SDK objects in parser ([#1312](#1312)) ([#1336](#1336)) ([5986c22](5986c22)) * **bedrock:** add boto3 1.41 + CRT for aws login credentials ([#1486](#1486)) ([4db3bc9](4db3bc9)) * bump codebase-memory-mcp to v0.8.1 ([#1284](#1284)) ([530318b](530318b)) * **ccr:** make headroom_retrieve a hash-only full-content lookup ([#1532](#1532)) ([c2fc4d3](c2fc4d3)) * **ccr:** propagate --no-ccr-marker flag to all compressors ([#1022](#1022)) ([#1197](#1197)) ([0c9b42a](0c9b42a)) * **ccr:** skip Anthropic marker emission when tool injection is deferred ([#1273](#1273)) ([2cae13d](2cae13d)) * **ci:** extend gitleaks allowlist to cover test fixtures + verified examples ([#1539](#1539)) ([d2565a6](d2565a6)) * **ci:** guarantee model present in test shards to end cache-miss flakiness ([#1399](#1399)) ([2e29c72](2e29c72)) * **ci:** normalize Windows CRLF line endings in PR governance script ([#1012](#1012)) ([5194388](5194388)) * **cli:** add explicit UTF-8 encoding to file I/O in wrap commands ([#1126](#1126)) ([#1164](#1164)) ([a0cb798](a0cb798)) * **cli:** fall back gracefully when embedding-server sidecar is absent ([#1206](#1206)) ([38f1404](38f1404)) * **cli:** harden all CLI surfaces + fix docs accuracy ([#1491](#1491)) ([bd76235](bd76235)) * **cli:** wire --http2/--no-http2 (HEADROOM_HTTP2) into proxy command ([#1373](#1373)) ([e06b616](e06b616)) * **cli:** wire --rpm/--tpm and HEADROOM_RPM/HEADROOM_TPM to the Click proxy command ([#1375](#1375)) ([8aab8f2](8aab8f2)) * **code:** slice tree-sitter byte offsets as UTF-8 ([#1332](#1332)) ([8238402](8238402)) * **code:** validate Python compressed syntax ([#1302](#1302)) ([cbd361d](cbd361d)) * **code:** verify a real parse in tree-sitter availability check ([#1231](#1231)) ([#1299](#1299)) ([5e0bb69](5e0bb69)) * **codex:** retag threads on init so Codex Desktop history stays visible ([#961](#961)) ([#1349](#1349)) ([e6bbc40](e6bbc40)) * **codex:** stop pinning Codex memory MCP to one project db ([#1269](#1269)) ([ad7993b](ad7993b)) * **dashboard:** include RTK stats in the historical tab ([#1324](#1324)) ([35939c3](35939c3)) * **deps:** remediate dependency CVEs and publish SBOM ([#1509](#1509)) ([5771a80](5771a80)) * **docker:** persist session history across container revisions ([#1118](#1118)) ([5912d65](5912d65)) * **gemini:** offload compression to the executor ([#1382](#1382)) ([615848e](615848e)) * **gemini:** resolve Google model capabilities through ModelRegistry ([#1276](#1276)) ([17ecad9](17ecad9)) * **install:** guard install_agent_ensure against duplicate runtime spawns ([#1301](#1301)) ([8da0b4e](8da0b4e)) * **install:** repair macOS launchd restart/start lifecycle ([#1290](#1290)) ([da1a397](da1a397)) * **install:** stop duplicating ENTRYPOINT in persistent-docker runtime command ([#833](#833)) ([#1348](#1348)) ([feedead](feedead)) * **io:** use UTF-8 with locale fallback and preserve line endings on config/text I/O ([#1498](#1498)) ([1baa04e](1baa04e)) * **kompress:** hard override keeps must-keep tokens regardless of model score ([#1400](#1400)) ([42612c8](42612c8)) * **langchain:** disable streaming on wrapped model during ainvoke() ([#1287](#1287)) ([3590046](3590046)) * **mcp:** register managed installs with a resolvable headroom command ([#1386](#1386)) ([22def93](22def93)) * **mcp:** report correct savings_percent in headroom_compress ([#1106](#1106)) ([f216e43](f216e43)) * **opencode:** write local MCP config ([#1381](#1381)) ([6c83790](6c83790)) * **packaging:** move hnswlib to optional [vector] extra so [all] needs no C++ toolchain ([#1499](#1499)) ([80fa086](80fa086)) * patch rtk hook script to use absolute path after register_claude_hooks ([#571](#571)) ([b618d2d](b618d2d)) * **perf:** surface RTK/CLI context-tool savings in perf and the session card ([#1433](#1433)) ([9362747](9362747)) * **proxy:** add --protect-tool-results to prevent lossy compression of exact-output Bash results ([#1374](#1374)) ([51d4bcf](51d4bcf)) * **proxy:** add an Anthropic buffered read-timeout override ([#1331](#1331)) ([3be2526](3be2526)) * **proxy:** add versionless Vertex AI routes for Claude Code compatibility ([#1321](#1321)) ([bb3e040](bb3e040)) * **proxy:** bind before eager preload so a hung compressor load can't block startup ([#1500](#1500)) ([d5ac07f](d5ac07f)) * **proxy:** build SSL contexts for custom CA bundles ([#1134](#1134)) ([561ba17](561ba17)) * **proxy:** forward request-id headers on the streaming path ([#1100](#1100)) ([#1258](#1258)) ([3d59df7](3d59df7)) * **proxy:** gate CCR retrieve/compress endpoints to loopback ([#1338](#1338)) ([acafb2d](acafb2d)) * **proxy:** honor force_kompress routing profile ([#996](#996)) ([b4682d6](b4682d6)) * **proxy:** keep large compression results on the critical path ([#296](#296)) ([#1352](#1352)) ([90734b6](90734b6)) * **proxy:** offload /v1/compress to the compression executor to stop blocking the loop ([#1501](#1501)) ([27e010e](27e010e)) * **proxy:** preserve Responses memory continuations with store=false ([#1103](#1103)) ([cdfeeac](cdfeeac)) * **proxy:** queue mid-turn user messages on non-Bedrock streaming path ([#1377](#1377)) ([b09f027](b09f027)) * **proxy:** register interceptor in explicit transforms list when HEADROOM_INTERCEPT_ENABLED ([#1376](#1376)) ([55c700c](55c700c)) * **proxy:** report real input tokens on streaming message_start ([#1132](#1132)) ([#1305](#1305)) ([70cc96a](70cc96a)) * **proxy:** retry upstream 429 with Retry-After on both forwarders ([#1329](#1329)) ([90bee89](90bee89)) * **proxy:** retry upstream 529 overloaded like 429 on both forwarders ([#1495](#1495)) ([547b15d](547b15d)) * **proxy:** stop re-compressing headroom_retrieve output and emitting unredeemable markers ([#1323](#1323)) ([43494ff](43494ff)) * **proxy:** strip Codex lite header from OpenAI WebSockets ([#1543](#1543)) ([5d3803a](5d3803a)) * **read-lifecycle:** persist STALE Read originals in the CCR store ([#1488](#1488)) ([9157173](9157173)) * recover persistent proxy feature checks and reject non-Copilot exchange URL ([#1465](#1465)) ([16c638b](16c638b)) * remove agents.md ([#1540](#1540)) ([a7d3360](a7d3360)) * respect COPILOT_PROVIDER_TYPE env var when provider_type is auto ([#549](#549)) ([24cf256](24cf256)) * restore token-mode compression on frozen prefixes ([#1489](#1489)) ([8e0dadf](8e0dadf)) * **router:** degrade to pure-Python detection on native panic ([#1123](#1123)) ([#1260](#1260)) ([a00fb67](a00fb67)) * **rtk:** stop hook registration timing out on a forked daemon ([#1314](#1314)) ([9758817](9758817)) * **smart-crusher:** honor enable_ccr_marker on the opaque-blob path ([#1130](#1130)) ([27d6f8e](27d6f8e)) * **subscription:** only reset 5h contribution on real rollover, not API jitter ([#1255](#1255)) ([8d6c175](8d6c175)) * **subscription:** run transcript token scan off the event loop ([#1263](#1263)) ([f03021f](f03021f)) * surface output reduction without a restart, and explain $0.00 savings on Python 3.14 ([#1296](#1296)) ([c30ec4c](c30ec4c)) * **tests:** reset whole headroom logger subtree so caplog stays deterministic ([#1117](#1117)) ([fda4670](fda4670)) * **tls:** add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection ([#1308](#1308)) ([#1341](#1341)) ([52068dd](52068dd)) * **tokenizers:** price CJK/Kana/Hangul at ~1 token per char in EstimatingTokenCounter ([#1093](#1093)) ([a35fe86](a35fe86)) * **transforms:** gate tool string output from lossy compression ([#1307](#1307)) ([#1387](#1387)) ([c6c921a](c6c921a)) * **websocket:** harden responses websocket origin handling ([#1481](#1481)) ([c632023](c632023)) * **windows:** pin UTF-8 encoding on text-mode subprocess calls ([#1311](#1311)) ([d633e81](d633e81)) * **wrap:** add Copilot unwrap command ([#1251](#1251)) ([b4fde0c](b4fde0c)) * **wrap:** isolate proxy stdio from proxy.log on Windows ([#1191](#1191)) ([959ab0d](959ab0d)) * **wrap:** keep agent savings opt-in ([#1294](#1294)) ([b829ceb](b829ceb)) * **wrap:** show the dashboard URL when the proxy is already running ([#1313](#1313)) ([b0146c4](b0146c4)) ### Performance Improvements * **compression:** take large cold-start contexts off the synchronous kompress path ([#1171](#1171)) ([#1298](#1298)) ([6c68ff4](6c68ff4)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
JerrettDavis
pushed a commit
to peterlodri-sec/headroom
that referenced
this pull request
Jul 14, 2026
…on card (headroomlabs-ai#1433) ## Description `headroom perf` read only `proxy.log` compression records, so RTK's savings — which live in RTK's own lifetime counter and never land in `proxy.log` — were **invisible**: perf reported "token savings" while silently dropping the entire CLI-filtering layer. The dashboard **Session** card likewise showed only the session-delta (≈0 right after a proxy restart), with no scope label and no lifetime figure. This surfaces RTK lifetime savings in `headroom perf` (text + JSON) and clarifies the dashboard Session card. It complements headroomlabs-ai#1324 (which added RTK to the Historical tab) by covering the two surfaces headroomlabs-ai#1324 didn't: `perf` and the live Session card. Closes # N/A — complements headroomlabs-ai#1324; no standalone issue. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/perf/analyzer.py`: `format_report` and `build_perf_summary` now attach RTK/CLI context-tool **lifetime** savings, sourced best-effort from `_get_context_tool_stats().lifetime` (the same source `/stats` and headroomlabs-ai#1324 use). Lifetime — not session — is the right scope for a one-shot CLI, since the proxy-session baseline `/stats` subtracts is meaningless out of process. Omitted entirely when no tool is installed or its stats can't be read, so the report degrades to proxy-only rather than erroring. - `headroom/dashboard/templates/dashboard.html`: the Session card now labels the RTK number **"this session"**, uses the real `session_savings_pct` (via a new `cliFilteringSessionPctDisplay` getter) instead of an ad-hoc share, and shows **lifetime** alongside it (new `cliFilteringLifetime` getter + row, hidden when 0). - `tests/test_perf_cli_filtering.py` (new): perf surfaces RTK in text + JSON; omits cleanly when the tool is absent. - `tests/test_rtk_session_savings.py` (new): exercises the real `_get_context_tool_stats()` plumbing to pin that session RTK savings are the **delta from the startup baseline**, and session `savings_pct` is derived from that delta — not RTK's lifetime-diluted average. - `tests/test_proxy_dashboard_stats_cache.py`: updated the Session-card label assertion and added one for the new lifetime row. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/perf/analyzer.py tests/test_perf_cli_filtering.py tests/test_rtk_session_savings.py tests/test_proxy_dashboard_stats_cache.py All checks passed! $ mypy headroom/perf/analyzer.py mypy: No issues found $ python -m pytest tests/test_perf_cli_filtering.py tests/test_rtk_session_savings.py tests/test_proxy_dashboard_stats_cache.py tests/test_owned_asset_encoding.py -q 17 passed, 1 skipped in 15.66s ``` ## Real Behavior Proof - Environment: macOS (darwin), Python 3.12, branch `fix/rtk-savings-perf-dashboard`, RTK v0.28.2. - Exact command / steps: `headroom perf` and `headroom perf --format json`. - Observed result: the text report now includes a section `RTK CLI Filtering (lifetime, all-time) — Tokens saved: 26,867,610 (68.8%), Commands: 8,023`, and the JSON output carries `"cli_filtering": {"tool":"rtk","label":"RTK","tokens_saved":26867610,"commands":8023,"savings_pct":68.8}`. Before this change, both omitted RTK entirely (perf's "Total saved" was proxy-compression only). The dashboard template renders the new "this session" / "lifetime" RTK rows (verified via `get_dashboard_html()` + substring test). - Not tested: live dashboard browser click-through (template loads and the new strings are asserted by the substring test); CSV output of `perf` (per-model table only, by design). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG: left to Release Please (the conventional `fix(perf):` commit generates the entry on merge), matching how the existing "Bug Fixes" entries are produced. - Follow-up: headroomlabs-ai#1403 (`fix/rtk-savings-scope-regression`) bundles unrelated kompress must-keep work (overlaps headroomlabs-ai#1400/headroomlabs-ai#1419) and only documents the scope `%` invariant in the abstract. The real, code-exercising session-delta regression now lives here (`test_rtk_session_savings.py`), so headroomlabs-ai#1403 can be split — route the kompress bits to headroomlabs-ai#1400/headroomlabs-ai#1419 and drop the rest.
JerrettDavis
pushed a commit
to peterlodri-sec/headroom
that referenced
this pull request
Jul 14, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>0.28.0</summary> ## [0.28.0](headroomlabs-ai/headroom@v0.27.0...v0.28.0) (2026-06-29) ### Features * add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback ([headroomlabs-ai#1185](headroomlabs-ai#1185)) ([f309244](headroomlabs-ai@f309244)) * add first-class OpenCode support (wrap, learn, mcp install) ([headroomlabs-ai#559](headroomlabs-ai#559)) ([91cd210](headroomlabs-ai@91cd210)) * add HEADROOM_KEEPALIVE_EXPIRY to keep upstream connections warm ([headroomlabs-ai#1124](headroomlabs-ai#1124)) ([85786b3](headroomlabs-ai@85786b3)) * **azure-foundry:** derive upstream URL from ANTHROPIC_FOUNDRY_RESOURCE ([headroomlabs-ai#1138](headroomlabs-ai#1138)) ([e5031b0](headroomlabs-ai@e5031b0)) * **cache:** attribute prompt-cache misses to TTL lapse vs prefix change ([headroomlabs-ai#1313](headroomlabs-ai#1313)) ([headroomlabs-ai#1343](headroomlabs-ai#1343)) ([4658721](headroomlabs-ai@4658721)) * **code:** add Perl support to code-aware compressor ([headroomlabs-ai#1125](headroomlabs-ai#1125)) ([f39858c](headroomlabs-ai@f39858c)) * headroom wrap opencode / unwrap opencode CLI ([headroomlabs-ai#1105](headroomlabs-ai#1105)) ([b4571cc](headroomlabs-ai@b4571cc)) * **learn:** weight loops in Headroom Learn + RTK-loop eval ([headroomlabs-ai#1160](headroomlabs-ai#1160)) ([14e8dc4](headroomlabs-ai@14e8dc4)) * **learn:** write per-project learnings to CLAUDE.local.md by default ([headroomlabs-ai#1115](headroomlabs-ai#1115)) ([ced75e4](headroomlabs-ai@ced75e4)) * **proxy:** add request timeout config ([headroomlabs-ai#738](headroomlabs-ai#738)) ([c0745d4](headroomlabs-ai@c0745d4)) * **proxy:** pilot hardening — inbound auth, security headers, audit log, air-gap switch ([headroomlabs-ai#1537](headroomlabs-ai#1537)) ([546ab55](headroomlabs-ai@546ab55)) * **proxy:** support glob patterns in exclude_tools ([headroomlabs-ai#870](headroomlabs-ai#870)) ([headroomlabs-ai#1259](headroomlabs-ai#1259)) ([a2159c0](headroomlabs-ai@a2159c0)) * **read-maturation:** activity-based hold-back Read maturation (Mechanism B) ([headroomlabs-ai#1068](headroomlabs-ai#1068)) ([723b80c](headroomlabs-ai@723b80c)) * **savings:** durable savings ledger + headroom savings command ([headroomlabs-ai#1127](headroomlabs-ai#1127)) ([978ffa0](headroomlabs-ai@978ffa0)) * **wrap:** add --1m to preserve the 1M context window on wrap claude ([headroomlabs-ai#1158](headroomlabs-ai#1158)) ([headroomlabs-ai#1351](headroomlabs-ai#1351)) ([b50d9c1](headroomlabs-ai@b50d9c1)) * **wrap:** make tokensave the primary coding-task compressor, Serena the backup ([headroomlabs-ai#1230](headroomlabs-ai#1230)) ([dca9853](headroomlabs-ai@dca9853)) ### Bug Fixes * **agent-evals:** Phase 0 — coding-agent accuracy A/B framework ([headroomlabs-ai#1037](headroomlabs-ai#1037)) ([84f9871](headroomlabs-ai@84f9871)) * **agno:** tolerate streaming tool-call SDK objects in parser ([headroomlabs-ai#1312](headroomlabs-ai#1312)) ([headroomlabs-ai#1336](headroomlabs-ai#1336)) ([5986c22](headroomlabs-ai@5986c22)) * **bedrock:** add boto3 1.41 + CRT for aws login credentials ([headroomlabs-ai#1486](headroomlabs-ai#1486)) ([4db3bc9](headroomlabs-ai@4db3bc9)) * bump codebase-memory-mcp to v0.8.1 ([headroomlabs-ai#1284](headroomlabs-ai#1284)) ([530318b](headroomlabs-ai@530318b)) * **ccr:** make headroom_retrieve a hash-only full-content lookup ([headroomlabs-ai#1532](headroomlabs-ai#1532)) ([c2fc4d3](headroomlabs-ai@c2fc4d3)) * **ccr:** propagate --no-ccr-marker flag to all compressors ([headroomlabs-ai#1022](headroomlabs-ai#1022)) ([headroomlabs-ai#1197](headroomlabs-ai#1197)) ([0c9b42a](headroomlabs-ai@0c9b42a)) * **ccr:** skip Anthropic marker emission when tool injection is deferred ([headroomlabs-ai#1273](headroomlabs-ai#1273)) ([2cae13d](headroomlabs-ai@2cae13d)) * **ci:** extend gitleaks allowlist to cover test fixtures + verified examples ([headroomlabs-ai#1539](headroomlabs-ai#1539)) ([d2565a6](headroomlabs-ai@d2565a6)) * **ci:** guarantee model present in test shards to end cache-miss flakiness ([headroomlabs-ai#1399](headroomlabs-ai#1399)) ([2e29c72](headroomlabs-ai@2e29c72)) * **ci:** normalize Windows CRLF line endings in PR governance script ([headroomlabs-ai#1012](headroomlabs-ai#1012)) ([5194388](headroomlabs-ai@5194388)) * **cli:** add explicit UTF-8 encoding to file I/O in wrap commands ([headroomlabs-ai#1126](headroomlabs-ai#1126)) ([headroomlabs-ai#1164](headroomlabs-ai#1164)) ([a0cb798](headroomlabs-ai@a0cb798)) * **cli:** fall back gracefully when embedding-server sidecar is absent ([headroomlabs-ai#1206](headroomlabs-ai#1206)) ([38f1404](headroomlabs-ai@38f1404)) * **cli:** harden all CLI surfaces + fix docs accuracy ([headroomlabs-ai#1491](headroomlabs-ai#1491)) ([bd76235](headroomlabs-ai@bd76235)) * **cli:** wire --http2/--no-http2 (HEADROOM_HTTP2) into proxy command ([headroomlabs-ai#1373](headroomlabs-ai#1373)) ([e06b616](headroomlabs-ai@e06b616)) * **cli:** wire --rpm/--tpm and HEADROOM_RPM/HEADROOM_TPM to the Click proxy command ([headroomlabs-ai#1375](headroomlabs-ai#1375)) ([8aab8f2](headroomlabs-ai@8aab8f2)) * **code:** slice tree-sitter byte offsets as UTF-8 ([headroomlabs-ai#1332](headroomlabs-ai#1332)) ([8238402](headroomlabs-ai@8238402)) * **code:** validate Python compressed syntax ([headroomlabs-ai#1302](headroomlabs-ai#1302)) ([cbd361d](headroomlabs-ai@cbd361d)) * **code:** verify a real parse in tree-sitter availability check ([headroomlabs-ai#1231](headroomlabs-ai#1231)) ([headroomlabs-ai#1299](headroomlabs-ai#1299)) ([5e0bb69](headroomlabs-ai@5e0bb69)) * **codex:** retag threads on init so Codex Desktop history stays visible ([headroomlabs-ai#961](headroomlabs-ai#961)) ([headroomlabs-ai#1349](headroomlabs-ai#1349)) ([e6bbc40](headroomlabs-ai@e6bbc40)) * **codex:** stop pinning Codex memory MCP to one project db ([headroomlabs-ai#1269](headroomlabs-ai#1269)) ([ad7993b](headroomlabs-ai@ad7993b)) * **dashboard:** include RTK stats in the historical tab ([headroomlabs-ai#1324](headroomlabs-ai#1324)) ([35939c3](headroomlabs-ai@35939c3)) * **deps:** remediate dependency CVEs and publish SBOM ([headroomlabs-ai#1509](headroomlabs-ai#1509)) ([5771a80](headroomlabs-ai@5771a80)) * **docker:** persist session history across container revisions ([headroomlabs-ai#1118](headroomlabs-ai#1118)) ([5912d65](headroomlabs-ai@5912d65)) * **gemini:** offload compression to the executor ([headroomlabs-ai#1382](headroomlabs-ai#1382)) ([615848e](headroomlabs-ai@615848e)) * **gemini:** resolve Google model capabilities through ModelRegistry ([headroomlabs-ai#1276](headroomlabs-ai#1276)) ([17ecad9](headroomlabs-ai@17ecad9)) * **install:** guard install_agent_ensure against duplicate runtime spawns ([headroomlabs-ai#1301](headroomlabs-ai#1301)) ([8da0b4e](headroomlabs-ai@8da0b4e)) * **install:** repair macOS launchd restart/start lifecycle ([headroomlabs-ai#1290](headroomlabs-ai#1290)) ([da1a397](headroomlabs-ai@da1a397)) * **install:** stop duplicating ENTRYPOINT in persistent-docker runtime command ([headroomlabs-ai#833](headroomlabs-ai#833)) ([headroomlabs-ai#1348](headroomlabs-ai#1348)) ([feedead](headroomlabs-ai@feedead)) * **io:** use UTF-8 with locale fallback and preserve line endings on config/text I/O ([headroomlabs-ai#1498](headroomlabs-ai#1498)) ([1baa04e](headroomlabs-ai@1baa04e)) * **kompress:** hard override keeps must-keep tokens regardless of model score ([headroomlabs-ai#1400](headroomlabs-ai#1400)) ([42612c8](headroomlabs-ai@42612c8)) * **langchain:** disable streaming on wrapped model during ainvoke() ([headroomlabs-ai#1287](headroomlabs-ai#1287)) ([3590046](headroomlabs-ai@3590046)) * **mcp:** register managed installs with a resolvable headroom command ([headroomlabs-ai#1386](headroomlabs-ai#1386)) ([22def93](headroomlabs-ai@22def93)) * **mcp:** report correct savings_percent in headroom_compress ([headroomlabs-ai#1106](headroomlabs-ai#1106)) ([f216e43](headroomlabs-ai@f216e43)) * **opencode:** write local MCP config ([headroomlabs-ai#1381](headroomlabs-ai#1381)) ([6c83790](headroomlabs-ai@6c83790)) * **packaging:** move hnswlib to optional [vector] extra so [all] needs no C++ toolchain ([headroomlabs-ai#1499](headroomlabs-ai#1499)) ([80fa086](headroomlabs-ai@80fa086)) * patch rtk hook script to use absolute path after register_claude_hooks ([headroomlabs-ai#571](headroomlabs-ai#571)) ([b618d2d](headroomlabs-ai@b618d2d)) * **perf:** surface RTK/CLI context-tool savings in perf and the session card ([headroomlabs-ai#1433](headroomlabs-ai#1433)) ([9362747](headroomlabs-ai@9362747)) * **proxy:** add --protect-tool-results to prevent lossy compression of exact-output Bash results ([headroomlabs-ai#1374](headroomlabs-ai#1374)) ([51d4bcf](headroomlabs-ai@51d4bcf)) * **proxy:** add an Anthropic buffered read-timeout override ([headroomlabs-ai#1331](headroomlabs-ai#1331)) ([3be2526](headroomlabs-ai@3be2526)) * **proxy:** add versionless Vertex AI routes for Claude Code compatibility ([headroomlabs-ai#1321](headroomlabs-ai#1321)) ([bb3e040](headroomlabs-ai@bb3e040)) * **proxy:** bind before eager preload so a hung compressor load can't block startup ([headroomlabs-ai#1500](headroomlabs-ai#1500)) ([d5ac07f](headroomlabs-ai@d5ac07f)) * **proxy:** build SSL contexts for custom CA bundles ([headroomlabs-ai#1134](headroomlabs-ai#1134)) ([561ba17](headroomlabs-ai@561ba17)) * **proxy:** forward request-id headers on the streaming path ([headroomlabs-ai#1100](headroomlabs-ai#1100)) ([headroomlabs-ai#1258](headroomlabs-ai#1258)) ([3d59df7](headroomlabs-ai@3d59df7)) * **proxy:** gate CCR retrieve/compress endpoints to loopback ([headroomlabs-ai#1338](headroomlabs-ai#1338)) ([acafb2d](headroomlabs-ai@acafb2d)) * **proxy:** honor force_kompress routing profile ([headroomlabs-ai#996](headroomlabs-ai#996)) ([b4682d6](headroomlabs-ai@b4682d6)) * **proxy:** keep large compression results on the critical path ([headroomlabs-ai#296](headroomlabs-ai#296)) ([headroomlabs-ai#1352](headroomlabs-ai#1352)) ([90734b6](headroomlabs-ai@90734b6)) * **proxy:** offload /v1/compress to the compression executor to stop blocking the loop ([headroomlabs-ai#1501](headroomlabs-ai#1501)) ([27e010e](headroomlabs-ai@27e010e)) * **proxy:** preserve Responses memory continuations with store=false ([headroomlabs-ai#1103](headroomlabs-ai#1103)) ([cdfeeac](headroomlabs-ai@cdfeeac)) * **proxy:** queue mid-turn user messages on non-Bedrock streaming path ([headroomlabs-ai#1377](headroomlabs-ai#1377)) ([b09f027](headroomlabs-ai@b09f027)) * **proxy:** register interceptor in explicit transforms list when HEADROOM_INTERCEPT_ENABLED ([headroomlabs-ai#1376](headroomlabs-ai#1376)) ([55c700c](headroomlabs-ai@55c700c)) * **proxy:** report real input tokens on streaming message_start ([headroomlabs-ai#1132](headroomlabs-ai#1132)) ([headroomlabs-ai#1305](headroomlabs-ai#1305)) ([70cc96a](headroomlabs-ai@70cc96a)) * **proxy:** retry upstream 429 with Retry-After on both forwarders ([headroomlabs-ai#1329](headroomlabs-ai#1329)) ([90bee89](headroomlabs-ai@90bee89)) * **proxy:** retry upstream 529 overloaded like 429 on both forwarders ([headroomlabs-ai#1495](headroomlabs-ai#1495)) ([547b15d](headroomlabs-ai@547b15d)) * **proxy:** stop re-compressing headroom_retrieve output and emitting unredeemable markers ([headroomlabs-ai#1323](headroomlabs-ai#1323)) ([43494ff](headroomlabs-ai@43494ff)) * **proxy:** strip Codex lite header from OpenAI WebSockets ([headroomlabs-ai#1543](headroomlabs-ai#1543)) ([5d3803a](headroomlabs-ai@5d3803a)) * **read-lifecycle:** persist STALE Read originals in the CCR store ([headroomlabs-ai#1488](headroomlabs-ai#1488)) ([9157173](headroomlabs-ai@9157173)) * recover persistent proxy feature checks and reject non-Copilot exchange URL ([headroomlabs-ai#1465](headroomlabs-ai#1465)) ([16c638b](headroomlabs-ai@16c638b)) * remove agents.md ([headroomlabs-ai#1540](headroomlabs-ai#1540)) ([a7d3360](headroomlabs-ai@a7d3360)) * respect COPILOT_PROVIDER_TYPE env var when provider_type is auto ([headroomlabs-ai#549](headroomlabs-ai#549)) ([24cf256](headroomlabs-ai@24cf256)) * restore token-mode compression on frozen prefixes ([headroomlabs-ai#1489](headroomlabs-ai#1489)) ([8e0dadf](headroomlabs-ai@8e0dadf)) * **router:** degrade to pure-Python detection on native panic ([headroomlabs-ai#1123](headroomlabs-ai#1123)) ([headroomlabs-ai#1260](headroomlabs-ai#1260)) ([a00fb67](headroomlabs-ai@a00fb67)) * **rtk:** stop hook registration timing out on a forked daemon ([headroomlabs-ai#1314](headroomlabs-ai#1314)) ([9758817](headroomlabs-ai@9758817)) * **smart-crusher:** honor enable_ccr_marker on the opaque-blob path ([headroomlabs-ai#1130](headroomlabs-ai#1130)) ([27d6f8e](headroomlabs-ai@27d6f8e)) * **subscription:** only reset 5h contribution on real rollover, not API jitter ([headroomlabs-ai#1255](headroomlabs-ai#1255)) ([8d6c175](headroomlabs-ai@8d6c175)) * **subscription:** run transcript token scan off the event loop ([headroomlabs-ai#1263](headroomlabs-ai#1263)) ([f03021f](headroomlabs-ai@f03021f)) * surface output reduction without a restart, and explain $0.00 savings on Python 3.14 ([headroomlabs-ai#1296](headroomlabs-ai#1296)) ([c30ec4c](headroomlabs-ai@c30ec4c)) * **tests:** reset whole headroom logger subtree so caplog stays deterministic ([headroomlabs-ai#1117](headroomlabs-ai#1117)) ([fda4670](headroomlabs-ai@fda4670)) * **tls:** add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection ([headroomlabs-ai#1308](headroomlabs-ai#1308)) ([headroomlabs-ai#1341](headroomlabs-ai#1341)) ([52068dd](headroomlabs-ai@52068dd)) * **tokenizers:** price CJK/Kana/Hangul at ~1 token per char in EstimatingTokenCounter ([headroomlabs-ai#1093](headroomlabs-ai#1093)) ([a35fe86](headroomlabs-ai@a35fe86)) * **transforms:** gate tool string output from lossy compression ([headroomlabs-ai#1307](headroomlabs-ai#1307)) ([headroomlabs-ai#1387](headroomlabs-ai#1387)) ([c6c921a](headroomlabs-ai@c6c921a)) * **websocket:** harden responses websocket origin handling ([headroomlabs-ai#1481](headroomlabs-ai#1481)) ([c632023](headroomlabs-ai@c632023)) * **windows:** pin UTF-8 encoding on text-mode subprocess calls ([headroomlabs-ai#1311](headroomlabs-ai#1311)) ([d633e81](headroomlabs-ai@d633e81)) * **wrap:** add Copilot unwrap command ([headroomlabs-ai#1251](headroomlabs-ai#1251)) ([b4fde0c](headroomlabs-ai@b4fde0c)) * **wrap:** isolate proxy stdio from proxy.log on Windows ([headroomlabs-ai#1191](headroomlabs-ai#1191)) ([959ab0d](headroomlabs-ai@959ab0d)) * **wrap:** keep agent savings opt-in ([headroomlabs-ai#1294](headroomlabs-ai#1294)) ([b829ceb](headroomlabs-ai@b829ceb)) * **wrap:** show the dashboard URL when the proxy is already running ([headroomlabs-ai#1313](headroomlabs-ai#1313)) ([b0146c4](headroomlabs-ai@b0146c4)) ### Performance Improvements * **compression:** take large cold-start contexts off the synchronous kompress path ([headroomlabs-ai#1171](headroomlabs-ai#1171)) ([headroomlabs-ai#1298](headroomlabs-ai#1298)) ([6c68ff4](headroomlabs-ai@6c68ff4)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
headroom perfread onlyproxy.logcompression records, so RTK's savings — which live in RTK's own lifetime counter and never land inproxy.log— were invisible: perf reported "token savings" while silently dropping the entire CLI-filtering layer. The dashboard Session card likewise showed only the session-delta (≈0 right after a proxy restart), with no scope label and no lifetime figure.This surfaces RTK lifetime savings in
headroom perf(text + JSON) and clarifies the dashboard Session card. It complements #1324 (which added RTK to the Historical tab) by covering the two surfaces #1324 didn't:perfand the live Session card.Closes # N/A — complements #1324; no standalone issue.
Type of Change
Changes Made
headroom/perf/analyzer.py:format_reportandbuild_perf_summarynow attach RTK/CLI context-tool lifetime savings, sourced best-effort from_get_context_tool_stats().lifetime(the same source/statsand fix(dashboard): include RTK stats in the historical tab #1324 use). Lifetime — not session — is the right scope for a one-shot CLI, since the proxy-session baseline/statssubtracts is meaningless out of process. Omitted entirely when no tool is installed or its stats can't be read, so the report degrades to proxy-only rather than erroring.headroom/dashboard/templates/dashboard.html: the Session card now labels the RTK number "this session", uses the realsession_savings_pct(via a newcliFilteringSessionPctDisplaygetter) instead of an ad-hoc share, and shows lifetime alongside it (newcliFilteringLifetimegetter + row, hidden when 0).tests/test_perf_cli_filtering.py(new): perf surfaces RTK in text + JSON; omits cleanly when the tool is absent.tests/test_rtk_session_savings.py(new): exercises the real_get_context_tool_stats()plumbing to pin that session RTK savings are the delta from the startup baseline, and sessionsavings_pctis derived from that delta — not RTK's lifetime-diluted average.tests/test_proxy_dashboard_stats_cache.py: updated the Session-card label assertion and added one for the new lifetime row.Testing
pytest)ruff check)mypy headroom)Test Output
Real Behavior Proof
fix/rtk-savings-perf-dashboard, RTK v0.28.2.headroom perfandheadroom perf --format json.RTK CLI Filtering (lifetime, all-time) — Tokens saved: 26,867,610 (68.8%), Commands: 8,023,and the JSON output carries
"cli_filtering": {"tool":"rtk","label":"RTK","tokens_saved":26867610,"commands":8023,"savings_pct":68.8}. Before this change, both omitted RTK entirely (perf's "Total saved" was proxy-compression only). The dashboard template renders the new "this session" / "lifetime" RTK rows (verified viaget_dashboard_html()+ substring test).perf(per-model table only, by design).Review Readiness
Checklist
Additional Notes
fix(perf):commit generates the entry on merge), matching how the existing "Bug Fixes" entries are produced.fix/rtk-savings-scope-regression) bundles unrelated kompress must-keep work (overlaps fix(kompress): hard override keeps must-keep tokens regardless of model score #1400/feat(kompress): sliding-window must-keep override for subword-tokenized patterns #1419) and only documents the scope%invariant in the abstract. The real, code-exercising session-delta regression now lives here (test_rtk_session_savings.py), so test(rtk): document scope-mixing underreporting bug #1403 can be split — route the kompress bits to fix(kompress): hard override keeps must-keep tokens regardless of model score #1400/feat(kompress): sliding-window must-keep override for subword-tokenized patterns #1419 and drop the rest.