Skip to content

fix(dashboard): align token savings headline denominator#1653

Merged
JerrettDavis merged 1 commit into
headroomlabs-ai:mainfrom
chris-yyau:codex/fix-token-savings-headline
Jul 2, 2026
Merged

fix(dashboard): align token savings headline denominator#1653
JerrettDavis merged 1 commit into
headroomlabs-ai:mainfrom
chris-yyau:codex/fix-token-savings-headline

Conversation

@chris-yyau

@chris-yyau chris-yyau commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes a dashboard denominator mismatch in the Token Savings card.

The headline was showing the active attempted-token ratio, while the same card's sublabel reports total-wire savings. This made sessions show values like about 17% in the headline and about 1.2% in the total-wire line for the same saved-token count.

This changes the headline to use stats.tokens.savings_percent, with proxy_savings_percent as a fallback, so the headline and card copy use the same denominator.

Closes #

Type of Change

  • Bug fix (non-breaking change fixes issue)
  • New feature (non-breaking change adds functionality)
  • Breaking change (fix or feature cause existing functionality change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • Updated headlineSavingsPercent to prefer the total-wire savings_percent metric.
  • Updated the headline tooltip to say Of total wire input tokens.
  • Added a focused dashboard regression test that prevents the headline getter from using active_savings_percent or proxy_attempted_tokens.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added new functionality
  • Manual testing performed

Test Output

$ python3 - <<'PY'
from pathlib import Path
html = Path('headroom/dashboard/templates/dashboard.html').read_text(encoding='utf-8')
assert 'stats.tokens?.savings_percent' in html
assert 'Of total wire input tokens' in html
start = html.index('get headlineSavingsPercent()')
end = html.index('get headlineSavingsTitle()', start)
headline = html[start:end]
assert 'active_savings_percent' not in headline
assert 'proxy_attempted_tokens' not in headline
print('dashboard headline denominator check passed')
PY
dashboard headline denominator check passed

$ uv run --extra dev pytest tests/test_dashboard_token_savings.py
============================= test session starts ==============================
platform darwin -- Python 3.13.3, pytest-9.0.3, pluggy-1.6.0
collected 1 item
tests/test_dashboard_token_savings.py::test_token_savings_headline_uses_total_wire_denominator PASSED [100%]
============================== 1 passed in 0.10s ===============================

$ uv run --extra dev ruff check tests/test_dashboard_token_savings.py
All checks passed!

Real Behavior Proof

  • Environment: Local Headroom dashboard served from the installed 0.28.0 package on macOS, proxy on 127.0.0.1:8788, checked against the same dashboard template logic patched in this PR.
  • Exact command / steps: Queried local /stats?cached=1, compared tokens.active_savings_percent with tokens.savings_percent, patched the dashboard template locally, then refreshed /dashboard and confirmed the served headlineSavingsPercent getter reads tokens.savings_percent.
  • Observed result: Local stats showed active_savings_percent around 16.78 while tokens.savings_percent, tokens.proxy_savings_percent, and agent total savings were around 1.24. Before the patch, the dashboard headline used the 16.78 active value even though the card text said total wire. After the local template patch, the served dashboard getter uses the 1.24 total-wire value.
  • Not tested: Full cross-browser visual regression; this PR only changes the Alpine getter denominator and adds a source-level regression test.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows project's style guidelines
  • I performed self-review my code
  • I commented my code, particularly in hard-to-understand areas
  • I made corresponding changes documentation
  • My changes generate no new warnings
  • I added tests prove fix is effective or feature works
  • New and existing unit tests pass locally my changes
  • I updated CHANGELOG.md if applicable

Screenshots (if applicable)

N/A.

Additional Notes

  • Documentation and CHANGELOG are N/A for this narrow dashboard bug fix.
  • CI is green and the PR is ready for review.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: needs author action Pull request body or readiness checklist still needs author updates label Jul 1, 2026
@chris-yyau chris-yyau force-pushed the codex/fix-token-savings-headline branch from 0a382af to 6116449 Compare July 1, 2026 15:18
@chris-yyau chris-yyau changed the title [codex] Fix token savings dashboard headline fix(dashboard): align token savings headline denominator Jul 1, 2026
@github-actions github-actions Bot added status: needs author action Pull request body or readiness checklist still needs author updates and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jul 1, 2026
@chris-yyau chris-yyau marked this pull request as ready for review July 1, 2026 15:35
@github-actions github-actions Bot added status: needs author action Pull request body or readiness checklist still needs author updates status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jul 1, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The headline and tooltip now use the same total-wire denominator, and the focused template regression protects against reintroducing the active attempted-token denominator. The only non-success check I see is a cancelled label run, not a code failure.

@JerrettDavis JerrettDavis merged commit 646e705 into headroomlabs-ai:main Jul 2, 2026
44 of 45 checks passed
@github-actions github-actions Bot mentioned this pull request Jul 2, 2026
chopratejas pushed a commit that referenced this pull request Jul 3, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.29.0</summary>

##
[0.29.0](v0.28.0...v0.29.0)
(2026-07-03)


### Features

* **proxy:** add --lossless no-CCR mode with format-native compaction
([#1721](#1721))
([c75ebde](c75ebde))
* **stats:** surface Codex WS compression counters in /stats summary
([#1680](#1680))
([2fe19c3](2fe19c3))
* **transforms:** adaptive Otsu KEEP/DROP threshold (+ land relevance
split on main)
([#1726](#1726))
([eea667a](eea667a))


### Bug Fixes

* **bedrock:** fail fast when session-token auth lacks botocore
([#1553](#1553))
([54cfa36](54cfa36))
* **bedrock:** route ARNs via converse, named AWS profiles, and au. re…
([#1456](#1456))
([7d87aa2](7d87aa2))
* **ccr:** honor workspace dir for sqlite store
([#1564](#1564))
([96e1dfe](96e1dfe))
* **claude:** surface Remote Control proxy incompatibility
([#1610](#1610))
([4bf7f92](4bf7f92))
* **cli:** stop advertising unwired compression tuning env vars in
banner
([#1634](#1634))
([d5bf98d](d5bf98d))
* **codex:** avoid duplicate headroom provider config
([#1431](#1431))
([ddd4adf](ddd4adf))
* **compression:** reject lossy unmarked tool output in unit router path
([#1479](#1479))
([de24cd5](de24cd5))
* **cortex-code:** migrate to current Cortex REST API endpoints + add
e2e benchmarks
([#1474](#1474))
([f00ace6](f00ace6))
* **dashboard:** align token savings headline denominator
([#1653](#1653))
([646e705](646e705))
* **dashboard:** derive per-project setup URL from live origin
([#1511](#1511))
([e035aef](e035aef))
* **detection:** contain unidiff panic on orphaned +++ target line
([#1548](#1548))
([e386c09](e386c09))
* **evals:** CJK-aware F1 tokenization + token estimation
([#1527](#1527))
([99a8540](99a8540))
* **install:** close parent log fd in start_detached_agent
([#1576](#1576))
([816cb85](816cb85))
* **install:** use Windows-safe PID liveness probe in runtime_status
([#1544](#1544))
([#1560](#1560))
([6b227b9](6b227b9))
* **learn:** aggregate verbosity baselines across projects instead of
overwriting
([#1288](#1288))
([27a5468](27a5468))
* **mcp:** show lifetime totals and label rolling session scope in
headroom_stats
([#1428](#1428))
([1c0e152](1c0e152))
* **memory:** cap local embedder CPU thread oversubscription
([#198](#198))
([#1559](#1559))
([b84afbf](b84afbf))
* **memory:** singleflight LocalBackend init to stop cold-start races
([#1691](#1691))
([bec47a1](bec47a1))
* **openclaw:** detect uv-installed headroom binary in ~/.local/bin
([#1459](#1459))
([adaeb88](adaeb88))
* **opencode:** preserve custom OpenAI gateway paths
([#1596](#1596))
([c19347c](c19347c))
* **opencode:** route native providers + load transport plugin, fix
Serena context
([#1573](#1573))
([ad0034f](ad0034f))
* preserve anthropic passthrough tool order
([#1427](#1427))
([a932247](a932247))
* **proxy/auth:** match real Anthropic OAuth token prefix (sk-ant-oat)
([#1672](#1672))
([8cddf9b](8cddf9b))
* **proxy:** expose persistent savings metrics
([#1647](#1647))
([5fe4e7b](5fe4e7b))
* **proxy:** fail open when kompress saturation would exhaust
pre-upstream budget
([#1430](#1430))
([15ac650](15ac650))
* **proxy:** handle streaming CCR retrieval
([#1451](#1451))
([d337e3b](d337e3b))
* **proxy:** include system/tools/sampling in cache key
([#1473](#1473))
([312129a](312129a))
* **proxy:** preserve Responses passthrough bytes
([#1598](#1598))
([2a34a82](2a34a82))
* **proxy:** strip Codex lite header on the HTTP /responses path
([#1663](#1663))
([9fbd47b](9fbd47b))
* **proxy:** wire --compression-max-workers /
HEADROOM_COMPRESSION_MAX_WORKERS
([#1632](#1632))
([814ffa3](814ffa3))
* **savings:** count cache-read tokens in input cost estimate
([#1429](#1429))
([72ade37](72ade37))
* skip Magika backend on x86 CPUs without AVX2
([#1162](#1162))
([64783d8](64783d8))
* **transforms/content-router:** route grep/log output away from HTML
extractor
([#1719](#1719))
([0d18ef2](0d18ef2))
* **transforms:** bound native content detection with a Windows watchdog
([#575](#575))
([#1563](#1563))
([95abca3](95abca3))
* Vertex AI support for Claude Code with ANTHROPIC_VERTEX_BASE_URL
([#1393](#1393))
([cff7247](cff7247))
* **wrap:** detach the shared proxy on Windows so it survives an
ungraceful agent close
([#1464](#1464))
([6cba441](6cba441))
* **wrap:** preserve custom Vertex base URL
([#1477](#1477))
([75427bb](75427bb))
* **wrap:** remove rtk instructions from Codex AGENTS.md on unwrap
([#1604](#1604))
([c9d717c](c9d717c))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
JerrettDavis pushed a commit to peterlodri-sec/headroom that referenced this pull request Jul 14, 2026
…s-ai#1653)

## Description
Fixes a dashboard denominator mismatch in the Token Savings card.

The headline was showing the active attempted-token ratio, while the
same card's sublabel reports total-wire savings. This made sessions show
values like about 17% in the headline and about 1.2% in the total-wire
line for the same saved-token count.

This changes the headline to use `stats.tokens.savings_percent`, with
`proxy_savings_percent` as a fallback, so the headline and card copy use
the same denominator.

Closes #

## Type of Change
- [x] Bug fix (non-breaking change fixes issue)
- [ ] New feature (non-breaking change adds functionality)
- [ ] Breaking change (fix or feature cause existing functionality
change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made
- Updated `headlineSavingsPercent` to prefer the total-wire
`savings_percent` metric.
- Updated the headline tooltip to say `Of total wire input tokens`.
- Added a focused dashboard regression test that prevents the headline
getter from using `active_savings_percent` or `proxy_attempted_tokens`.

## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added new functionality
- [x] Manual testing performed

### Test Output

```text
$ python3 - <<'PY'
from pathlib import Path
html = Path('headroom/dashboard/templates/dashboard.html').read_text(encoding='utf-8')
assert 'stats.tokens?.savings_percent' in html
assert 'Of total wire input tokens' in html
start = html.index('get headlineSavingsPercent()')
end = html.index('get headlineSavingsTitle()', start)
headline = html[start:end]
assert 'active_savings_percent' not in headline
assert 'proxy_attempted_tokens' not in headline
print('dashboard headline denominator check passed')
PY
dashboard headline denominator check passed

$ uv run --extra dev pytest tests/test_dashboard_token_savings.py
============================= test session starts ==============================
platform darwin -- Python 3.13.3, pytest-9.0.3, pluggy-1.6.0
collected 1 item
tests/test_dashboard_token_savings.py::test_token_savings_headline_uses_total_wire_denominator PASSED [100%]
============================== 1 passed in 0.10s ===============================

$ uv run --extra dev ruff check tests/test_dashboard_token_savings.py
All checks passed!
```

## Real Behavior Proof
- Environment: Local Headroom dashboard served from the installed 0.28.0
package on macOS, proxy on `127.0.0.1:8788`, checked against the same
dashboard template logic patched in this PR.
- Exact command / steps: Queried local `/stats?cached=1`, compared
`tokens.active_savings_percent` with `tokens.savings_percent`, patched
the dashboard template locally, then refreshed `/dashboard` and
confirmed the served `headlineSavingsPercent` getter reads
`tokens.savings_percent`.
- Observed result: Local stats showed `active_savings_percent` around
16.78 while `tokens.savings_percent`, `tokens.proxy_savings_percent`,
and agent total savings were around 1.24. Before the patch, the
dashboard headline used the 16.78 active value even though the card text
said total wire. After the local template patch, the served dashboard
getter uses the 1.24 total-wire value.
- Not tested: Full cross-browser visual regression; this PR only changes
the Alpine getter denominator and adds a source-level regression test.

## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist
- [x] My code follows project's style guidelines
- [x] I performed self-review my code
- [x] I commented my code, particularly in hard-to-understand areas
- [ ] I made corresponding changes documentation
- [x] My changes generate no new warnings
- [x] I added tests prove fix is effective or feature works
- [x] New and existing unit tests pass locally my changes
- [ ] I updated CHANGELOG.md if applicable

## Screenshots (if applicable)
N/A.

## Additional Notes
- Documentation and CHANGELOG are N/A for this narrow dashboard bug fix.
- CI is green and the PR is ready for review.
JerrettDavis pushed a commit to peterlodri-sec/headroom that referenced this pull request Jul 14, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.29.0</summary>

##
[0.29.0](headroomlabs-ai/headroom@v0.28.0...v0.29.0)
(2026-07-03)


### Features

* **proxy:** add --lossless no-CCR mode with format-native compaction
([headroomlabs-ai#1721](headroomlabs-ai#1721))
([c75ebde](headroomlabs-ai@c75ebde))
* **stats:** surface Codex WS compression counters in /stats summary
([headroomlabs-ai#1680](headroomlabs-ai#1680))
([2fe19c3](headroomlabs-ai@2fe19c3))
* **transforms:** adaptive Otsu KEEP/DROP threshold (+ land relevance
split on main)
([headroomlabs-ai#1726](headroomlabs-ai#1726))
([eea667a](headroomlabs-ai@eea667a))


### Bug Fixes

* **bedrock:** fail fast when session-token auth lacks botocore
([headroomlabs-ai#1553](headroomlabs-ai#1553))
([54cfa36](headroomlabs-ai@54cfa36))
* **bedrock:** route ARNs via converse, named AWS profiles, and au. re…
([headroomlabs-ai#1456](headroomlabs-ai#1456))
([7d87aa2](headroomlabs-ai@7d87aa2))
* **ccr:** honor workspace dir for sqlite store
([headroomlabs-ai#1564](headroomlabs-ai#1564))
([96e1dfe](headroomlabs-ai@96e1dfe))
* **claude:** surface Remote Control proxy incompatibility
([headroomlabs-ai#1610](headroomlabs-ai#1610))
([4bf7f92](headroomlabs-ai@4bf7f92))
* **cli:** stop advertising unwired compression tuning env vars in
banner
([headroomlabs-ai#1634](headroomlabs-ai#1634))
([d5bf98d](headroomlabs-ai@d5bf98d))
* **codex:** avoid duplicate headroom provider config
([headroomlabs-ai#1431](headroomlabs-ai#1431))
([ddd4adf](headroomlabs-ai@ddd4adf))
* **compression:** reject lossy unmarked tool output in unit router path
([headroomlabs-ai#1479](headroomlabs-ai#1479))
([de24cd5](headroomlabs-ai@de24cd5))
* **cortex-code:** migrate to current Cortex REST API endpoints + add
e2e benchmarks
([headroomlabs-ai#1474](headroomlabs-ai#1474))
([f00ace6](headroomlabs-ai@f00ace6))
* **dashboard:** align token savings headline denominator
([headroomlabs-ai#1653](headroomlabs-ai#1653))
([646e705](headroomlabs-ai@646e705))
* **dashboard:** derive per-project setup URL from live origin
([headroomlabs-ai#1511](headroomlabs-ai#1511))
([e035aef](headroomlabs-ai@e035aef))
* **detection:** contain unidiff panic on orphaned +++ target line
([headroomlabs-ai#1548](headroomlabs-ai#1548))
([e386c09](headroomlabs-ai@e386c09))
* **evals:** CJK-aware F1 tokenization + token estimation
([headroomlabs-ai#1527](headroomlabs-ai#1527))
([99a8540](headroomlabs-ai@99a8540))
* **install:** close parent log fd in start_detached_agent
([headroomlabs-ai#1576](headroomlabs-ai#1576))
([816cb85](headroomlabs-ai@816cb85))
* **install:** use Windows-safe PID liveness probe in runtime_status
([headroomlabs-ai#1544](headroomlabs-ai#1544))
([headroomlabs-ai#1560](headroomlabs-ai#1560))
([6b227b9](headroomlabs-ai@6b227b9))
* **learn:** aggregate verbosity baselines across projects instead of
overwriting
([headroomlabs-ai#1288](headroomlabs-ai#1288))
([27a5468](headroomlabs-ai@27a5468))
* **mcp:** show lifetime totals and label rolling session scope in
headroom_stats
([headroomlabs-ai#1428](headroomlabs-ai#1428))
([1c0e152](headroomlabs-ai@1c0e152))
* **memory:** cap local embedder CPU thread oversubscription
([headroomlabs-ai#198](headroomlabs-ai#198))
([headroomlabs-ai#1559](headroomlabs-ai#1559))
([b84afbf](headroomlabs-ai@b84afbf))
* **memory:** singleflight LocalBackend init to stop cold-start races
([headroomlabs-ai#1691](headroomlabs-ai#1691))
([bec47a1](headroomlabs-ai@bec47a1))
* **openclaw:** detect uv-installed headroom binary in ~/.local/bin
([headroomlabs-ai#1459](headroomlabs-ai#1459))
([adaeb88](headroomlabs-ai@adaeb88))
* **opencode:** preserve custom OpenAI gateway paths
([headroomlabs-ai#1596](headroomlabs-ai#1596))
([c19347c](headroomlabs-ai@c19347c))
* **opencode:** route native providers + load transport plugin, fix
Serena context
([headroomlabs-ai#1573](headroomlabs-ai#1573))
([ad0034f](headroomlabs-ai@ad0034f))
* preserve anthropic passthrough tool order
([headroomlabs-ai#1427](headroomlabs-ai#1427))
([a932247](headroomlabs-ai@a932247))
* **proxy/auth:** match real Anthropic OAuth token prefix (sk-ant-oat)
([headroomlabs-ai#1672](headroomlabs-ai#1672))
([8cddf9b](headroomlabs-ai@8cddf9b))
* **proxy:** expose persistent savings metrics
([headroomlabs-ai#1647](headroomlabs-ai#1647))
([5fe4e7b](headroomlabs-ai@5fe4e7b))
* **proxy:** fail open when kompress saturation would exhaust
pre-upstream budget
([headroomlabs-ai#1430](headroomlabs-ai#1430))
([15ac650](headroomlabs-ai@15ac650))
* **proxy:** handle streaming CCR retrieval
([headroomlabs-ai#1451](headroomlabs-ai#1451))
([d337e3b](headroomlabs-ai@d337e3b))
* **proxy:** include system/tools/sampling in cache key
([headroomlabs-ai#1473](headroomlabs-ai#1473))
([312129a](headroomlabs-ai@312129a))
* **proxy:** preserve Responses passthrough bytes
([headroomlabs-ai#1598](headroomlabs-ai#1598))
([2a34a82](headroomlabs-ai@2a34a82))
* **proxy:** strip Codex lite header on the HTTP /responses path
([headroomlabs-ai#1663](headroomlabs-ai#1663))
([9fbd47b](headroomlabs-ai@9fbd47b))
* **proxy:** wire --compression-max-workers /
HEADROOM_COMPRESSION_MAX_WORKERS
([headroomlabs-ai#1632](headroomlabs-ai#1632))
([814ffa3](headroomlabs-ai@814ffa3))
* **savings:** count cache-read tokens in input cost estimate
([headroomlabs-ai#1429](headroomlabs-ai#1429))
([72ade37](headroomlabs-ai@72ade37))
* skip Magika backend on x86 CPUs without AVX2
([headroomlabs-ai#1162](headroomlabs-ai#1162))
([64783d8](headroomlabs-ai@64783d8))
* **transforms/content-router:** route grep/log output away from HTML
extractor
([headroomlabs-ai#1719](headroomlabs-ai#1719))
([0d18ef2](headroomlabs-ai@0d18ef2))
* **transforms:** bound native content detection with a Windows watchdog
([headroomlabs-ai#575](headroomlabs-ai#575))
([headroomlabs-ai#1563](headroomlabs-ai#1563))
([95abca3](headroomlabs-ai@95abca3))
* Vertex AI support for Claude Code with ANTHROPIC_VERTEX_BASE_URL
([headroomlabs-ai#1393](headroomlabs-ai#1393))
([cff7247](headroomlabs-ai@cff7247))
* **wrap:** detach the shared proxy on Windows so it survives an
ungraceful agent close
([headroomlabs-ai#1464](headroomlabs-ai#1464))
([6cba441](headroomlabs-ai@6cba441))
* **wrap:** preserve custom Vertex base URL
([headroomlabs-ai#1477](headroomlabs-ai#1477))
([75427bb](headroomlabs-ai@75427bb))
* **wrap:** remove rtk instructions from Codex AGENTS.md on unwrap
([headroomlabs-ai#1604](headroomlabs-ai#1604))
([c9d717c](headroomlabs-ai@c9d717c))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants