fix(dashboard): align token savings headline denominator#1653
Merged
JerrettDavis merged 1 commit intoJul 2, 2026
Merged
Conversation
Contributor
PR governanceThis PR follows the template and is marked ready for human review. |
0a382af to
6116449
Compare
JerrettDavis
approved these changes
Jul 1, 2026
JerrettDavis
left a comment
Collaborator
There was a problem hiding this comment.
Looks good. The headline and tooltip now use the same total-wire denominator, and the focused template regression protects against reintroducing the active attempted-token denominator. The only non-success check I see is a cancelled label run, not a code failure.
Merged
chopratejas
pushed a commit
that referenced
this pull request
Jul 3, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>0.29.0</summary> ## [0.29.0](v0.28.0...v0.29.0) (2026-07-03) ### Features * **proxy:** add --lossless no-CCR mode with format-native compaction ([#1721](#1721)) ([c75ebde](c75ebde)) * **stats:** surface Codex WS compression counters in /stats summary ([#1680](#1680)) ([2fe19c3](2fe19c3)) * **transforms:** adaptive Otsu KEEP/DROP threshold (+ land relevance split on main) ([#1726](#1726)) ([eea667a](eea667a)) ### Bug Fixes * **bedrock:** fail fast when session-token auth lacks botocore ([#1553](#1553)) ([54cfa36](54cfa36)) * **bedrock:** route ARNs via converse, named AWS profiles, and au. re… ([#1456](#1456)) ([7d87aa2](7d87aa2)) * **ccr:** honor workspace dir for sqlite store ([#1564](#1564)) ([96e1dfe](96e1dfe)) * **claude:** surface Remote Control proxy incompatibility ([#1610](#1610)) ([4bf7f92](4bf7f92)) * **cli:** stop advertising unwired compression tuning env vars in banner ([#1634](#1634)) ([d5bf98d](d5bf98d)) * **codex:** avoid duplicate headroom provider config ([#1431](#1431)) ([ddd4adf](ddd4adf)) * **compression:** reject lossy unmarked tool output in unit router path ([#1479](#1479)) ([de24cd5](de24cd5)) * **cortex-code:** migrate to current Cortex REST API endpoints + add e2e benchmarks ([#1474](#1474)) ([f00ace6](f00ace6)) * **dashboard:** align token savings headline denominator ([#1653](#1653)) ([646e705](646e705)) * **dashboard:** derive per-project setup URL from live origin ([#1511](#1511)) ([e035aef](e035aef)) * **detection:** contain unidiff panic on orphaned +++ target line ([#1548](#1548)) ([e386c09](e386c09)) * **evals:** CJK-aware F1 tokenization + token estimation ([#1527](#1527)) ([99a8540](99a8540)) * **install:** close parent log fd in start_detached_agent ([#1576](#1576)) ([816cb85](816cb85)) * **install:** use Windows-safe PID liveness probe in runtime_status ([#1544](#1544)) ([#1560](#1560)) ([6b227b9](6b227b9)) * **learn:** aggregate verbosity baselines across projects instead of overwriting ([#1288](#1288)) ([27a5468](27a5468)) * **mcp:** show lifetime totals and label rolling session scope in headroom_stats ([#1428](#1428)) ([1c0e152](1c0e152)) * **memory:** cap local embedder CPU thread oversubscription ([#198](#198)) ([#1559](#1559)) ([b84afbf](b84afbf)) * **memory:** singleflight LocalBackend init to stop cold-start races ([#1691](#1691)) ([bec47a1](bec47a1)) * **openclaw:** detect uv-installed headroom binary in ~/.local/bin ([#1459](#1459)) ([adaeb88](adaeb88)) * **opencode:** preserve custom OpenAI gateway paths ([#1596](#1596)) ([c19347c](c19347c)) * **opencode:** route native providers + load transport plugin, fix Serena context ([#1573](#1573)) ([ad0034f](ad0034f)) * preserve anthropic passthrough tool order ([#1427](#1427)) ([a932247](a932247)) * **proxy/auth:** match real Anthropic OAuth token prefix (sk-ant-oat) ([#1672](#1672)) ([8cddf9b](8cddf9b)) * **proxy:** expose persistent savings metrics ([#1647](#1647)) ([5fe4e7b](5fe4e7b)) * **proxy:** fail open when kompress saturation would exhaust pre-upstream budget ([#1430](#1430)) ([15ac650](15ac650)) * **proxy:** handle streaming CCR retrieval ([#1451](#1451)) ([d337e3b](d337e3b)) * **proxy:** include system/tools/sampling in cache key ([#1473](#1473)) ([312129a](312129a)) * **proxy:** preserve Responses passthrough bytes ([#1598](#1598)) ([2a34a82](2a34a82)) * **proxy:** strip Codex lite header on the HTTP /responses path ([#1663](#1663)) ([9fbd47b](9fbd47b)) * **proxy:** wire --compression-max-workers / HEADROOM_COMPRESSION_MAX_WORKERS ([#1632](#1632)) ([814ffa3](814ffa3)) * **savings:** count cache-read tokens in input cost estimate ([#1429](#1429)) ([72ade37](72ade37)) * skip Magika backend on x86 CPUs without AVX2 ([#1162](#1162)) ([64783d8](64783d8)) * **transforms/content-router:** route grep/log output away from HTML extractor ([#1719](#1719)) ([0d18ef2](0d18ef2)) * **transforms:** bound native content detection with a Windows watchdog ([#575](#575)) ([#1563](#1563)) ([95abca3](95abca3)) * Vertex AI support for Claude Code with ANTHROPIC_VERTEX_BASE_URL ([#1393](#1393)) ([cff7247](cff7247)) * **wrap:** detach the shared proxy on Windows so it survives an ungraceful agent close ([#1464](#1464)) ([6cba441](6cba441)) * **wrap:** preserve custom Vertex base URL ([#1477](#1477)) ([75427bb](75427bb)) * **wrap:** remove rtk instructions from Codex AGENTS.md on unwrap ([#1604](#1604)) ([c9d717c](c9d717c)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This was referenced Jul 12, 2026
JerrettDavis
pushed a commit
to peterlodri-sec/headroom
that referenced
this pull request
Jul 14, 2026
…s-ai#1653) ## Description Fixes a dashboard denominator mismatch in the Token Savings card. The headline was showing the active attempted-token ratio, while the same card's sublabel reports total-wire savings. This made sessions show values like about 17% in the headline and about 1.2% in the total-wire line for the same saved-token count. This changes the headline to use `stats.tokens.savings_percent`, with `proxy_savings_percent` as a fallback, so the headline and card copy use the same denominator. Closes # ## Type of Change - [x] Bug fix (non-breaking change fixes issue) - [ ] New feature (non-breaking change adds functionality) - [ ] Breaking change (fix or feature cause existing functionality change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Updated `headlineSavingsPercent` to prefer the total-wire `savings_percent` metric. - Updated the headline tooltip to say `Of total wire input tokens`. - Added a focused dashboard regression test that prevents the headline getter from using `active_savings_percent` or `proxy_attempted_tokens`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added new functionality - [x] Manual testing performed ### Test Output ```text $ python3 - <<'PY' from pathlib import Path html = Path('headroom/dashboard/templates/dashboard.html').read_text(encoding='utf-8') assert 'stats.tokens?.savings_percent' in html assert 'Of total wire input tokens' in html start = html.index('get headlineSavingsPercent()') end = html.index('get headlineSavingsTitle()', start) headline = html[start:end] assert 'active_savings_percent' not in headline assert 'proxy_attempted_tokens' not in headline print('dashboard headline denominator check passed') PY dashboard headline denominator check passed $ uv run --extra dev pytest tests/test_dashboard_token_savings.py ============================= test session starts ============================== platform darwin -- Python 3.13.3, pytest-9.0.3, pluggy-1.6.0 collected 1 item tests/test_dashboard_token_savings.py::test_token_savings_headline_uses_total_wire_denominator PASSED [100%] ============================== 1 passed in 0.10s =============================== $ uv run --extra dev ruff check tests/test_dashboard_token_savings.py All checks passed! ``` ## Real Behavior Proof - Environment: Local Headroom dashboard served from the installed 0.28.0 package on macOS, proxy on `127.0.0.1:8788`, checked against the same dashboard template logic patched in this PR. - Exact command / steps: Queried local `/stats?cached=1`, compared `tokens.active_savings_percent` with `tokens.savings_percent`, patched the dashboard template locally, then refreshed `/dashboard` and confirmed the served `headlineSavingsPercent` getter reads `tokens.savings_percent`. - Observed result: Local stats showed `active_savings_percent` around 16.78 while `tokens.savings_percent`, `tokens.proxy_savings_percent`, and agent total savings were around 1.24. Before the patch, the dashboard headline used the 16.78 active value even though the card text said total wire. After the local template patch, the served dashboard getter uses the 1.24 total-wire value. - Not tested: Full cross-browser visual regression; this PR only changes the Alpine getter denominator and adds a source-level regression test. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows project's style guidelines - [x] I performed self-review my code - [x] I commented my code, particularly in hard-to-understand areas - [ ] I made corresponding changes documentation - [x] My changes generate no new warnings - [x] I added tests prove fix is effective or feature works - [x] New and existing unit tests pass locally my changes - [ ] I updated CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - Documentation and CHANGELOG are N/A for this narrow dashboard bug fix. - CI is green and the PR is ready for review.
JerrettDavis
pushed a commit
to peterlodri-sec/headroom
that referenced
this pull request
Jul 14, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>0.29.0</summary> ## [0.29.0](headroomlabs-ai/headroom@v0.28.0...v0.29.0) (2026-07-03) ### Features * **proxy:** add --lossless no-CCR mode with format-native compaction ([headroomlabs-ai#1721](headroomlabs-ai#1721)) ([c75ebde](headroomlabs-ai@c75ebde)) * **stats:** surface Codex WS compression counters in /stats summary ([headroomlabs-ai#1680](headroomlabs-ai#1680)) ([2fe19c3](headroomlabs-ai@2fe19c3)) * **transforms:** adaptive Otsu KEEP/DROP threshold (+ land relevance split on main) ([headroomlabs-ai#1726](headroomlabs-ai#1726)) ([eea667a](headroomlabs-ai@eea667a)) ### Bug Fixes * **bedrock:** fail fast when session-token auth lacks botocore ([headroomlabs-ai#1553](headroomlabs-ai#1553)) ([54cfa36](headroomlabs-ai@54cfa36)) * **bedrock:** route ARNs via converse, named AWS profiles, and au. re… ([headroomlabs-ai#1456](headroomlabs-ai#1456)) ([7d87aa2](headroomlabs-ai@7d87aa2)) * **ccr:** honor workspace dir for sqlite store ([headroomlabs-ai#1564](headroomlabs-ai#1564)) ([96e1dfe](headroomlabs-ai@96e1dfe)) * **claude:** surface Remote Control proxy incompatibility ([headroomlabs-ai#1610](headroomlabs-ai#1610)) ([4bf7f92](headroomlabs-ai@4bf7f92)) * **cli:** stop advertising unwired compression tuning env vars in banner ([headroomlabs-ai#1634](headroomlabs-ai#1634)) ([d5bf98d](headroomlabs-ai@d5bf98d)) * **codex:** avoid duplicate headroom provider config ([headroomlabs-ai#1431](headroomlabs-ai#1431)) ([ddd4adf](headroomlabs-ai@ddd4adf)) * **compression:** reject lossy unmarked tool output in unit router path ([headroomlabs-ai#1479](headroomlabs-ai#1479)) ([de24cd5](headroomlabs-ai@de24cd5)) * **cortex-code:** migrate to current Cortex REST API endpoints + add e2e benchmarks ([headroomlabs-ai#1474](headroomlabs-ai#1474)) ([f00ace6](headroomlabs-ai@f00ace6)) * **dashboard:** align token savings headline denominator ([headroomlabs-ai#1653](headroomlabs-ai#1653)) ([646e705](headroomlabs-ai@646e705)) * **dashboard:** derive per-project setup URL from live origin ([headroomlabs-ai#1511](headroomlabs-ai#1511)) ([e035aef](headroomlabs-ai@e035aef)) * **detection:** contain unidiff panic on orphaned +++ target line ([headroomlabs-ai#1548](headroomlabs-ai#1548)) ([e386c09](headroomlabs-ai@e386c09)) * **evals:** CJK-aware F1 tokenization + token estimation ([headroomlabs-ai#1527](headroomlabs-ai#1527)) ([99a8540](headroomlabs-ai@99a8540)) * **install:** close parent log fd in start_detached_agent ([headroomlabs-ai#1576](headroomlabs-ai#1576)) ([816cb85](headroomlabs-ai@816cb85)) * **install:** use Windows-safe PID liveness probe in runtime_status ([headroomlabs-ai#1544](headroomlabs-ai#1544)) ([headroomlabs-ai#1560](headroomlabs-ai#1560)) ([6b227b9](headroomlabs-ai@6b227b9)) * **learn:** aggregate verbosity baselines across projects instead of overwriting ([headroomlabs-ai#1288](headroomlabs-ai#1288)) ([27a5468](headroomlabs-ai@27a5468)) * **mcp:** show lifetime totals and label rolling session scope in headroom_stats ([headroomlabs-ai#1428](headroomlabs-ai#1428)) ([1c0e152](headroomlabs-ai@1c0e152)) * **memory:** cap local embedder CPU thread oversubscription ([headroomlabs-ai#198](headroomlabs-ai#198)) ([headroomlabs-ai#1559](headroomlabs-ai#1559)) ([b84afbf](headroomlabs-ai@b84afbf)) * **memory:** singleflight LocalBackend init to stop cold-start races ([headroomlabs-ai#1691](headroomlabs-ai#1691)) ([bec47a1](headroomlabs-ai@bec47a1)) * **openclaw:** detect uv-installed headroom binary in ~/.local/bin ([headroomlabs-ai#1459](headroomlabs-ai#1459)) ([adaeb88](headroomlabs-ai@adaeb88)) * **opencode:** preserve custom OpenAI gateway paths ([headroomlabs-ai#1596](headroomlabs-ai#1596)) ([c19347c](headroomlabs-ai@c19347c)) * **opencode:** route native providers + load transport plugin, fix Serena context ([headroomlabs-ai#1573](headroomlabs-ai#1573)) ([ad0034f](headroomlabs-ai@ad0034f)) * preserve anthropic passthrough tool order ([headroomlabs-ai#1427](headroomlabs-ai#1427)) ([a932247](headroomlabs-ai@a932247)) * **proxy/auth:** match real Anthropic OAuth token prefix (sk-ant-oat) ([headroomlabs-ai#1672](headroomlabs-ai#1672)) ([8cddf9b](headroomlabs-ai@8cddf9b)) * **proxy:** expose persistent savings metrics ([headroomlabs-ai#1647](headroomlabs-ai#1647)) ([5fe4e7b](headroomlabs-ai@5fe4e7b)) * **proxy:** fail open when kompress saturation would exhaust pre-upstream budget ([headroomlabs-ai#1430](headroomlabs-ai#1430)) ([15ac650](headroomlabs-ai@15ac650)) * **proxy:** handle streaming CCR retrieval ([headroomlabs-ai#1451](headroomlabs-ai#1451)) ([d337e3b](headroomlabs-ai@d337e3b)) * **proxy:** include system/tools/sampling in cache key ([headroomlabs-ai#1473](headroomlabs-ai#1473)) ([312129a](headroomlabs-ai@312129a)) * **proxy:** preserve Responses passthrough bytes ([headroomlabs-ai#1598](headroomlabs-ai#1598)) ([2a34a82](headroomlabs-ai@2a34a82)) * **proxy:** strip Codex lite header on the HTTP /responses path ([headroomlabs-ai#1663](headroomlabs-ai#1663)) ([9fbd47b](headroomlabs-ai@9fbd47b)) * **proxy:** wire --compression-max-workers / HEADROOM_COMPRESSION_MAX_WORKERS ([headroomlabs-ai#1632](headroomlabs-ai#1632)) ([814ffa3](headroomlabs-ai@814ffa3)) * **savings:** count cache-read tokens in input cost estimate ([headroomlabs-ai#1429](headroomlabs-ai#1429)) ([72ade37](headroomlabs-ai@72ade37)) * skip Magika backend on x86 CPUs without AVX2 ([headroomlabs-ai#1162](headroomlabs-ai#1162)) ([64783d8](headroomlabs-ai@64783d8)) * **transforms/content-router:** route grep/log output away from HTML extractor ([headroomlabs-ai#1719](headroomlabs-ai#1719)) ([0d18ef2](headroomlabs-ai@0d18ef2)) * **transforms:** bound native content detection with a Windows watchdog ([headroomlabs-ai#575](headroomlabs-ai#575)) ([headroomlabs-ai#1563](headroomlabs-ai#1563)) ([95abca3](headroomlabs-ai@95abca3)) * Vertex AI support for Claude Code with ANTHROPIC_VERTEX_BASE_URL ([headroomlabs-ai#1393](headroomlabs-ai#1393)) ([cff7247](headroomlabs-ai@cff7247)) * **wrap:** detach the shared proxy on Windows so it survives an ungraceful agent close ([headroomlabs-ai#1464](headroomlabs-ai#1464)) ([6cba441](headroomlabs-ai@6cba441)) * **wrap:** preserve custom Vertex base URL ([headroomlabs-ai#1477](headroomlabs-ai#1477)) ([75427bb](headroomlabs-ai@75427bb)) * **wrap:** remove rtk instructions from Codex AGENTS.md on unwrap ([headroomlabs-ai#1604](headroomlabs-ai#1604)) ([c9d717c](headroomlabs-ai@c9d717c)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes a dashboard denominator mismatch in the Token Savings card.
The headline was showing the active attempted-token ratio, while the same card's sublabel reports total-wire savings. This made sessions show values like about 17% in the headline and about 1.2% in the total-wire line for the same saved-token count.
This changes the headline to use
stats.tokens.savings_percent, withproxy_savings_percentas a fallback, so the headline and card copy use the same denominator.Closes #
Type of Change
Changes Made
headlineSavingsPercentto prefer the total-wiresavings_percentmetric.Of total wire input tokens.active_savings_percentorproxy_attempted_tokens.Testing
pytest)ruff check .)mypy headroom)Test Output
Real Behavior Proof
127.0.0.1:8788, checked against the same dashboard template logic patched in this PR./stats?cached=1, comparedtokens.active_savings_percentwithtokens.savings_percent, patched the dashboard template locally, then refreshed/dashboardand confirmed the servedheadlineSavingsPercentgetter readstokens.savings_percent.active_savings_percentaround 16.78 whiletokens.savings_percent,tokens.proxy_savings_percent, and agent total savings were around 1.24. Before the patch, the dashboard headline used the 16.78 active value even though the card text said total wire. After the local template patch, the served dashboard getter uses the 1.24 total-wire value.Review Readiness
Checklist
Screenshots (if applicable)
N/A.
Additional Notes