Skip to content

fix(code): slice tree-sitter byte offsets as UTF-8#1332

Merged
JerrettDavis merged 1 commit into
headroomlabs-ai:mainfrom
aivinay:fix-code-compressor-unicode-offsets
Jun 23, 2026
Merged

fix(code): slice tree-sitter byte offsets as UTF-8#1332
JerrettDavis merged 1 commit into
headroomlabs-ai:mainfrom
aivinay:fix-code-compressor-unicode-offsets

Conversation

@aivinay

@aivinay aivinay commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Description

CodeAwareCompressor was slicing Python strings with tree-sitter start_byte / end_byte offsets directly. That works for ASCII-only files, but it corrupts slices after non-ASCII source text such as CJK characters or emoji because tree-sitter offsets are UTF-8 byte offsets while Python string indexes are character offsets.

This caused code-aware compression to produce invalid intermediate Python and then safely fall back to the original file, resulting in 0% compression on affected files.

Closes #1319

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • Added _slice_code_bytes() in headroom/transforms/code_compressor.py to slice source text using UTF-8 byte offsets.
  • Updated _get_node_text() to use byte-safe slicing.
  • Routed the other direct tree-sitter byte-offset slices through the same helper.
  • Added regression tests in tests/test_transforms/test_code_compressor.py:
    • test_get_node_text_uses_utf8_byte_offsets
    • test_ast_compresses_python_after_non_ascii_source

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

$ .venv/bin/python -m pytest tests/test_transforms/test_code_compressor.py
68 passed, 1 warning

$ .venv/bin/python -m ruff check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py
All checks passed!

$ .venv/bin/python -m ruff format --check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py
2 files already formatted

$ git diff --check
# no output

$ /tmp/headroom-1319-venv/bin/python -m mypy headroom
headroom/proxy/server.py:1186: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
headroom/proxy/server.py:1257: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
headroom/proxy/server.py:1261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
pyproject.toml: note: unused section(s): module = ['mlx.*']
Success: no issues found in 394 source files

Real Behavior Proof

  • Environment: macOS arm64, Python 3.14.5, tree-sitter 0.25.2, tree-sitter-language-pack 0.13.0
  • Exact command / steps: On main, ran a local reproducer with a Python source string containing a CJK docstring before a second function; called _get_node_text() on the second tree-sitter function node; ran a full CodeAwareCompressor.compress(...) repro with non-ASCII module text before an import and a compressible function; re-ran both repros on this branch.
  • Observed result: Before fix, _get_node_text() returned the wrong slice ('nd():\n return 2\n' instead of 'def second():\n return 2') and full compression fell back to the original file with compression_ratio: 1.0; after fix, _get_node_text() returns the full expected function slice and full compression succeeds with compression_ratio < 1.0, syntax_valid: True, and does not return the original.
  • Not tested: Full repository test suite; live proxy/provider integrations; Windows/Linux platform-specific behavior.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the CHANGELOG.md if applicable

Screenshots (if applicable)

N/A

Additional Notes

  • Documentation was not updated because this is an internal bug fix with no user-facing API or behavior change beyond restoring intended compression.
  • CHANGELOG.md was not updated because the fix is narrow and issue-scoped; maintainers can advise if they want a changelog entry.
  • The fix is intentionally small and targeted: it only changes how tree-sitter byte offsets are converted back into Python source text, without changing compression heuristics or language behavior.

@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added status: needs author action Pull request body or readiness checklist still needs author updates status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jun 23, 2026
@aivinay

aivinay commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

@JerrettDavis Thanks for the approval! CI is only failing in unrelated memory/HuggingFace cache tests. Could you please rerun the failed jobs?

@JerrettDavis JerrettDavis merged commit 8238402 into headroomlabs-ai:main Jun 23, 2026
23 of 25 checks passed
@github-actions github-actions Bot mentioned this pull request Jun 25, 2026
chopratejas pushed a commit that referenced this pull request Jun 29, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.28.0</summary>

##
[0.28.0](v0.27.0...v0.28.0)
(2026-06-29)


### Features

* add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback
([#1185](#1185))
([f309244](f309244))
* add first-class OpenCode support (wrap, learn, mcp install)
([#559](#559))
([91cd210](91cd210))
* add HEADROOM_KEEPALIVE_EXPIRY to keep upstream connections warm
([#1124](#1124))
([85786b3](85786b3))
* **azure-foundry:** derive upstream URL from ANTHROPIC_FOUNDRY_RESOURCE
([#1138](#1138))
([e5031b0](e5031b0))
* **cache:** attribute prompt-cache misses to TTL lapse vs prefix change
([#1313](#1313))
([#1343](#1343))
([4658721](4658721))
* **code:** add Perl support to code-aware compressor
([#1125](#1125))
([f39858c](f39858c))
* headroom wrap opencode / unwrap opencode CLI
([#1105](#1105))
([b4571cc](b4571cc))
* **learn:** weight loops in Headroom Learn + RTK-loop eval
([#1160](#1160))
([14e8dc4](14e8dc4))
* **learn:** write per-project learnings to CLAUDE.local.md by default
([#1115](#1115))
([ced75e4](ced75e4))
* **proxy:** add request timeout config
([#738](#738))
([c0745d4](c0745d4))
* **proxy:** pilot hardening — inbound auth, security headers, audit
log, air-gap switch
([#1537](#1537))
([546ab55](546ab55))
* **proxy:** support glob patterns in exclude_tools
([#870](#870))
([#1259](#1259))
([a2159c0](a2159c0))
* **read-maturation:** activity-based hold-back Read maturation
(Mechanism B)
([#1068](#1068))
([723b80c](723b80c))
* **savings:** durable savings ledger + headroom savings command
([#1127](#1127))
([978ffa0](978ffa0))
* **wrap:** add --1m to preserve the 1M context window on wrap claude
([#1158](#1158))
([#1351](#1351))
([b50d9c1](b50d9c1))
* **wrap:** make tokensave the primary coding-task compressor, Serena
the backup
([#1230](#1230))
([dca9853](dca9853))


### Bug Fixes

* **agent-evals:** Phase 0 — coding-agent accuracy A/B framework
([#1037](#1037))
([84f9871](84f9871))
* **agno:** tolerate streaming tool-call SDK objects in parser
([#1312](#1312))
([#1336](#1336))
([5986c22](5986c22))
* **bedrock:** add boto3 1.41 + CRT for aws login credentials
([#1486](#1486))
([4db3bc9](4db3bc9))
* bump codebase-memory-mcp to v0.8.1
([#1284](#1284))
([530318b](530318b))
* **ccr:** make headroom_retrieve a hash-only full-content lookup
([#1532](#1532))
([c2fc4d3](c2fc4d3))
* **ccr:** propagate --no-ccr-marker flag to all compressors
([#1022](#1022))
([#1197](#1197))
([0c9b42a](0c9b42a))
* **ccr:** skip Anthropic marker emission when tool injection is
deferred
([#1273](#1273))
([2cae13d](2cae13d))
* **ci:** extend gitleaks allowlist to cover test fixtures + verified
examples
([#1539](#1539))
([d2565a6](d2565a6))
* **ci:** guarantee model present in test shards to end cache-miss
flakiness
([#1399](#1399))
([2e29c72](2e29c72))
* **ci:** normalize Windows CRLF line endings in PR governance script
([#1012](#1012))
([5194388](5194388))
* **cli:** add explicit UTF-8 encoding to file I/O in wrap commands
([#1126](#1126))
([#1164](#1164))
([a0cb798](a0cb798))
* **cli:** fall back gracefully when embedding-server sidecar is absent
([#1206](#1206))
([38f1404](38f1404))
* **cli:** harden all CLI surfaces + fix docs accuracy
([#1491](#1491))
([bd76235](bd76235))
* **cli:** wire --http2/--no-http2 (HEADROOM_HTTP2) into proxy command
([#1373](#1373))
([e06b616](e06b616))
* **cli:** wire --rpm/--tpm and HEADROOM_RPM/HEADROOM_TPM to the Click
proxy command
([#1375](#1375))
([8aab8f2](8aab8f2))
* **code:** slice tree-sitter byte offsets as UTF-8
([#1332](#1332))
([8238402](8238402))
* **code:** validate Python compressed syntax
([#1302](#1302))
([cbd361d](cbd361d))
* **code:** verify a real parse in tree-sitter availability check
([#1231](#1231))
([#1299](#1299))
([5e0bb69](5e0bb69))
* **codex:** retag threads on init so Codex Desktop history stays
visible ([#961](#961))
([#1349](#1349))
([e6bbc40](e6bbc40))
* **codex:** stop pinning Codex memory MCP to one project db
([#1269](#1269))
([ad7993b](ad7993b))
* **dashboard:** include RTK stats in the historical tab
([#1324](#1324))
([35939c3](35939c3))
* **deps:** remediate dependency CVEs and publish SBOM
([#1509](#1509))
([5771a80](5771a80))
* **docker:** persist session history across container revisions
([#1118](#1118))
([5912d65](5912d65))
* **gemini:** offload compression to the executor
([#1382](#1382))
([615848e](615848e))
* **gemini:** resolve Google model capabilities through ModelRegistry
([#1276](#1276))
([17ecad9](17ecad9))
* **install:** guard install_agent_ensure against duplicate runtime
spawns
([#1301](#1301))
([8da0b4e](8da0b4e))
* **install:** repair macOS launchd restart/start lifecycle
([#1290](#1290))
([da1a397](da1a397))
* **install:** stop duplicating ENTRYPOINT in persistent-docker runtime
command ([#833](#833))
([#1348](#1348))
([feedead](feedead))
* **io:** use UTF-8 with locale fallback and preserve line endings on
config/text I/O
([#1498](#1498))
([1baa04e](1baa04e))
* **kompress:** hard override keeps must-keep tokens regardless of model
score ([#1400](#1400))
([42612c8](42612c8))
* **langchain:** disable streaming on wrapped model during ainvoke()
([#1287](#1287))
([3590046](3590046))
* **mcp:** register managed installs with a resolvable headroom command
([#1386](#1386))
([22def93](22def93))
* **mcp:** report correct savings_percent in headroom_compress
([#1106](#1106))
([f216e43](f216e43))
* **opencode:** write local MCP config
([#1381](#1381))
([6c83790](6c83790))
* **packaging:** move hnswlib to optional [vector] extra so [all] needs
no C++ toolchain
([#1499](#1499))
([80fa086](80fa086))
* patch rtk hook script to use absolute path after register_claude_hooks
([#571](#571))
([b618d2d](b618d2d))
* **perf:** surface RTK/CLI context-tool savings in perf and the session
card ([#1433](#1433))
([9362747](9362747))
* **proxy:** add --protect-tool-results to prevent lossy compression of
exact-output Bash results
([#1374](#1374))
([51d4bcf](51d4bcf))
* **proxy:** add an Anthropic buffered read-timeout override
([#1331](#1331))
([3be2526](3be2526))
* **proxy:** add versionless Vertex AI routes for Claude Code
compatibility
([#1321](#1321))
([bb3e040](bb3e040))
* **proxy:** bind before eager preload so a hung compressor load can't
block startup
([#1500](#1500))
([d5ac07f](d5ac07f))
* **proxy:** build SSL contexts for custom CA bundles
([#1134](#1134))
([561ba17](561ba17))
* **proxy:** forward request-id headers on the streaming path
([#1100](#1100))
([#1258](#1258))
([3d59df7](3d59df7))
* **proxy:** gate CCR retrieve/compress endpoints to loopback
([#1338](#1338))
([acafb2d](acafb2d))
* **proxy:** honor force_kompress routing profile
([#996](#996))
([b4682d6](b4682d6))
* **proxy:** keep large compression results on the critical path
([#296](#296))
([#1352](#1352))
([90734b6](90734b6))
* **proxy:** offload /v1/compress to the compression executor to stop
blocking the loop
([#1501](#1501))
([27e010e](27e010e))
* **proxy:** preserve Responses memory continuations with store=false
([#1103](#1103))
([cdfeeac](cdfeeac))
* **proxy:** queue mid-turn user messages on non-Bedrock streaming path
([#1377](#1377))
([b09f027](b09f027))
* **proxy:** register interceptor in explicit transforms list when
HEADROOM_INTERCEPT_ENABLED
([#1376](#1376))
([55c700c](55c700c))
* **proxy:** report real input tokens on streaming message_start
([#1132](#1132))
([#1305](#1305))
([70cc96a](70cc96a))
* **proxy:** retry upstream 429 with Retry-After on both forwarders
([#1329](#1329))
([90bee89](90bee89))
* **proxy:** retry upstream 529 overloaded like 429 on both forwarders
([#1495](#1495))
([547b15d](547b15d))
* **proxy:** stop re-compressing headroom_retrieve output and emitting
unredeemable markers
([#1323](#1323))
([43494ff](43494ff))
* **proxy:** strip Codex lite header from OpenAI WebSockets
([#1543](#1543))
([5d3803a](5d3803a))
* **read-lifecycle:** persist STALE Read originals in the CCR store
([#1488](#1488))
([9157173](9157173))
* recover persistent proxy feature checks and reject non-Copilot
exchange URL
([#1465](#1465))
([16c638b](16c638b))
* remove agents.md
([#1540](#1540))
([a7d3360](a7d3360))
* respect COPILOT_PROVIDER_TYPE env var when provider_type is auto
([#549](#549))
([24cf256](24cf256))
* restore token-mode compression on frozen prefixes
([#1489](#1489))
([8e0dadf](8e0dadf))
* **router:** degrade to pure-Python detection on native panic
([#1123](#1123))
([#1260](#1260))
([a00fb67](a00fb67))
* **rtk:** stop hook registration timing out on a forked daemon
([#1314](#1314))
([9758817](9758817))
* **smart-crusher:** honor enable_ccr_marker on the opaque-blob path
([#1130](#1130))
([27d6f8e](27d6f8e))
* **subscription:** only reset 5h contribution on real rollover, not API
jitter
([#1255](#1255))
([8d6c175](8d6c175))
* **subscription:** run transcript token scan off the event loop
([#1263](#1263))
([f03021f](f03021f))
* surface output reduction without a restart, and explain $0.00 savings
on Python 3.14
([#1296](#1296))
([c30ec4c](c30ec4c))
* **tests:** reset whole headroom logger subtree so caplog stays
deterministic
([#1117](#1117))
([fda4670](fda4670))
* **tls:** add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection
([#1308](#1308))
([#1341](#1341))
([52068dd](52068dd))
* **tokenizers:** price CJK/Kana/Hangul at ~1 token per char in
EstimatingTokenCounter
([#1093](#1093))
([a35fe86](a35fe86))
* **transforms:** gate tool string output from lossy compression
([#1307](#1307))
([#1387](#1387))
([c6c921a](c6c921a))
* **websocket:** harden responses websocket origin handling
([#1481](#1481))
([c632023](c632023))
* **windows:** pin UTF-8 encoding on text-mode subprocess calls
([#1311](#1311))
([d633e81](d633e81))
* **wrap:** add Copilot unwrap command
([#1251](#1251))
([b4fde0c](b4fde0c))
* **wrap:** isolate proxy stdio from proxy.log on Windows
([#1191](#1191))
([959ab0d](959ab0d))
* **wrap:** keep agent savings opt-in
([#1294](#1294))
([b829ceb](b829ceb))
* **wrap:** show the dashboard URL when the proxy is already running
([#1313](#1313))
([b0146c4](b0146c4))


### Performance Improvements

* **compression:** take large cold-start contexts off the synchronous
kompress path
([#1171](#1171))
([#1298](#1298))
([6c68ff4](6c68ff4))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
JerrettDavis pushed a commit to peterlodri-sec/headroom that referenced this pull request Jul 14, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.28.0</summary>

##
[0.28.0](headroomlabs-ai/headroom@v0.27.0...v0.28.0)
(2026-06-29)


### Features

* add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback
([headroomlabs-ai#1185](headroomlabs-ai#1185))
([f309244](headroomlabs-ai@f309244))
* add first-class OpenCode support (wrap, learn, mcp install)
([headroomlabs-ai#559](headroomlabs-ai#559))
([91cd210](headroomlabs-ai@91cd210))
* add HEADROOM_KEEPALIVE_EXPIRY to keep upstream connections warm
([headroomlabs-ai#1124](headroomlabs-ai#1124))
([85786b3](headroomlabs-ai@85786b3))
* **azure-foundry:** derive upstream URL from ANTHROPIC_FOUNDRY_RESOURCE
([headroomlabs-ai#1138](headroomlabs-ai#1138))
([e5031b0](headroomlabs-ai@e5031b0))
* **cache:** attribute prompt-cache misses to TTL lapse vs prefix change
([headroomlabs-ai#1313](headroomlabs-ai#1313))
([headroomlabs-ai#1343](headroomlabs-ai#1343))
([4658721](headroomlabs-ai@4658721))
* **code:** add Perl support to code-aware compressor
([headroomlabs-ai#1125](headroomlabs-ai#1125))
([f39858c](headroomlabs-ai@f39858c))
* headroom wrap opencode / unwrap opencode CLI
([headroomlabs-ai#1105](headroomlabs-ai#1105))
([b4571cc](headroomlabs-ai@b4571cc))
* **learn:** weight loops in Headroom Learn + RTK-loop eval
([headroomlabs-ai#1160](headroomlabs-ai#1160))
([14e8dc4](headroomlabs-ai@14e8dc4))
* **learn:** write per-project learnings to CLAUDE.local.md by default
([headroomlabs-ai#1115](headroomlabs-ai#1115))
([ced75e4](headroomlabs-ai@ced75e4))
* **proxy:** add request timeout config
([headroomlabs-ai#738](headroomlabs-ai#738))
([c0745d4](headroomlabs-ai@c0745d4))
* **proxy:** pilot hardening — inbound auth, security headers, audit
log, air-gap switch
([headroomlabs-ai#1537](headroomlabs-ai#1537))
([546ab55](headroomlabs-ai@546ab55))
* **proxy:** support glob patterns in exclude_tools
([headroomlabs-ai#870](headroomlabs-ai#870))
([headroomlabs-ai#1259](headroomlabs-ai#1259))
([a2159c0](headroomlabs-ai@a2159c0))
* **read-maturation:** activity-based hold-back Read maturation
(Mechanism B)
([headroomlabs-ai#1068](headroomlabs-ai#1068))
([723b80c](headroomlabs-ai@723b80c))
* **savings:** durable savings ledger + headroom savings command
([headroomlabs-ai#1127](headroomlabs-ai#1127))
([978ffa0](headroomlabs-ai@978ffa0))
* **wrap:** add --1m to preserve the 1M context window on wrap claude
([headroomlabs-ai#1158](headroomlabs-ai#1158))
([headroomlabs-ai#1351](headroomlabs-ai#1351))
([b50d9c1](headroomlabs-ai@b50d9c1))
* **wrap:** make tokensave the primary coding-task compressor, Serena
the backup
([headroomlabs-ai#1230](headroomlabs-ai#1230))
([dca9853](headroomlabs-ai@dca9853))


### Bug Fixes

* **agent-evals:** Phase 0 — coding-agent accuracy A/B framework
([headroomlabs-ai#1037](headroomlabs-ai#1037))
([84f9871](headroomlabs-ai@84f9871))
* **agno:** tolerate streaming tool-call SDK objects in parser
([headroomlabs-ai#1312](headroomlabs-ai#1312))
([headroomlabs-ai#1336](headroomlabs-ai#1336))
([5986c22](headroomlabs-ai@5986c22))
* **bedrock:** add boto3 1.41 + CRT for aws login credentials
([headroomlabs-ai#1486](headroomlabs-ai#1486))
([4db3bc9](headroomlabs-ai@4db3bc9))
* bump codebase-memory-mcp to v0.8.1
([headroomlabs-ai#1284](headroomlabs-ai#1284))
([530318b](headroomlabs-ai@530318b))
* **ccr:** make headroom_retrieve a hash-only full-content lookup
([headroomlabs-ai#1532](headroomlabs-ai#1532))
([c2fc4d3](headroomlabs-ai@c2fc4d3))
* **ccr:** propagate --no-ccr-marker flag to all compressors
([headroomlabs-ai#1022](headroomlabs-ai#1022))
([headroomlabs-ai#1197](headroomlabs-ai#1197))
([0c9b42a](headroomlabs-ai@0c9b42a))
* **ccr:** skip Anthropic marker emission when tool injection is
deferred
([headroomlabs-ai#1273](headroomlabs-ai#1273))
([2cae13d](headroomlabs-ai@2cae13d))
* **ci:** extend gitleaks allowlist to cover test fixtures + verified
examples
([headroomlabs-ai#1539](headroomlabs-ai#1539))
([d2565a6](headroomlabs-ai@d2565a6))
* **ci:** guarantee model present in test shards to end cache-miss
flakiness
([headroomlabs-ai#1399](headroomlabs-ai#1399))
([2e29c72](headroomlabs-ai@2e29c72))
* **ci:** normalize Windows CRLF line endings in PR governance script
([headroomlabs-ai#1012](headroomlabs-ai#1012))
([5194388](headroomlabs-ai@5194388))
* **cli:** add explicit UTF-8 encoding to file I/O in wrap commands
([headroomlabs-ai#1126](headroomlabs-ai#1126))
([headroomlabs-ai#1164](headroomlabs-ai#1164))
([a0cb798](headroomlabs-ai@a0cb798))
* **cli:** fall back gracefully when embedding-server sidecar is absent
([headroomlabs-ai#1206](headroomlabs-ai#1206))
([38f1404](headroomlabs-ai@38f1404))
* **cli:** harden all CLI surfaces + fix docs accuracy
([headroomlabs-ai#1491](headroomlabs-ai#1491))
([bd76235](headroomlabs-ai@bd76235))
* **cli:** wire --http2/--no-http2 (HEADROOM_HTTP2) into proxy command
([headroomlabs-ai#1373](headroomlabs-ai#1373))
([e06b616](headroomlabs-ai@e06b616))
* **cli:** wire --rpm/--tpm and HEADROOM_RPM/HEADROOM_TPM to the Click
proxy command
([headroomlabs-ai#1375](headroomlabs-ai#1375))
([8aab8f2](headroomlabs-ai@8aab8f2))
* **code:** slice tree-sitter byte offsets as UTF-8
([headroomlabs-ai#1332](headroomlabs-ai#1332))
([8238402](headroomlabs-ai@8238402))
* **code:** validate Python compressed syntax
([headroomlabs-ai#1302](headroomlabs-ai#1302))
([cbd361d](headroomlabs-ai@cbd361d))
* **code:** verify a real parse in tree-sitter availability check
([headroomlabs-ai#1231](headroomlabs-ai#1231))
([headroomlabs-ai#1299](headroomlabs-ai#1299))
([5e0bb69](headroomlabs-ai@5e0bb69))
* **codex:** retag threads on init so Codex Desktop history stays
visible ([headroomlabs-ai#961](headroomlabs-ai#961))
([headroomlabs-ai#1349](headroomlabs-ai#1349))
([e6bbc40](headroomlabs-ai@e6bbc40))
* **codex:** stop pinning Codex memory MCP to one project db
([headroomlabs-ai#1269](headroomlabs-ai#1269))
([ad7993b](headroomlabs-ai@ad7993b))
* **dashboard:** include RTK stats in the historical tab
([headroomlabs-ai#1324](headroomlabs-ai#1324))
([35939c3](headroomlabs-ai@35939c3))
* **deps:** remediate dependency CVEs and publish SBOM
([headroomlabs-ai#1509](headroomlabs-ai#1509))
([5771a80](headroomlabs-ai@5771a80))
* **docker:** persist session history across container revisions
([headroomlabs-ai#1118](headroomlabs-ai#1118))
([5912d65](headroomlabs-ai@5912d65))
* **gemini:** offload compression to the executor
([headroomlabs-ai#1382](headroomlabs-ai#1382))
([615848e](headroomlabs-ai@615848e))
* **gemini:** resolve Google model capabilities through ModelRegistry
([headroomlabs-ai#1276](headroomlabs-ai#1276))
([17ecad9](headroomlabs-ai@17ecad9))
* **install:** guard install_agent_ensure against duplicate runtime
spawns
([headroomlabs-ai#1301](headroomlabs-ai#1301))
([8da0b4e](headroomlabs-ai@8da0b4e))
* **install:** repair macOS launchd restart/start lifecycle
([headroomlabs-ai#1290](headroomlabs-ai#1290))
([da1a397](headroomlabs-ai@da1a397))
* **install:** stop duplicating ENTRYPOINT in persistent-docker runtime
command ([headroomlabs-ai#833](headroomlabs-ai#833))
([headroomlabs-ai#1348](headroomlabs-ai#1348))
([feedead](headroomlabs-ai@feedead))
* **io:** use UTF-8 with locale fallback and preserve line endings on
config/text I/O
([headroomlabs-ai#1498](headroomlabs-ai#1498))
([1baa04e](headroomlabs-ai@1baa04e))
* **kompress:** hard override keeps must-keep tokens regardless of model
score ([headroomlabs-ai#1400](headroomlabs-ai#1400))
([42612c8](headroomlabs-ai@42612c8))
* **langchain:** disable streaming on wrapped model during ainvoke()
([headroomlabs-ai#1287](headroomlabs-ai#1287))
([3590046](headroomlabs-ai@3590046))
* **mcp:** register managed installs with a resolvable headroom command
([headroomlabs-ai#1386](headroomlabs-ai#1386))
([22def93](headroomlabs-ai@22def93))
* **mcp:** report correct savings_percent in headroom_compress
([headroomlabs-ai#1106](headroomlabs-ai#1106))
([f216e43](headroomlabs-ai@f216e43))
* **opencode:** write local MCP config
([headroomlabs-ai#1381](headroomlabs-ai#1381))
([6c83790](headroomlabs-ai@6c83790))
* **packaging:** move hnswlib to optional [vector] extra so [all] needs
no C++ toolchain
([headroomlabs-ai#1499](headroomlabs-ai#1499))
([80fa086](headroomlabs-ai@80fa086))
* patch rtk hook script to use absolute path after register_claude_hooks
([headroomlabs-ai#571](headroomlabs-ai#571))
([b618d2d](headroomlabs-ai@b618d2d))
* **perf:** surface RTK/CLI context-tool savings in perf and the session
card ([headroomlabs-ai#1433](headroomlabs-ai#1433))
([9362747](headroomlabs-ai@9362747))
* **proxy:** add --protect-tool-results to prevent lossy compression of
exact-output Bash results
([headroomlabs-ai#1374](headroomlabs-ai#1374))
([51d4bcf](headroomlabs-ai@51d4bcf))
* **proxy:** add an Anthropic buffered read-timeout override
([headroomlabs-ai#1331](headroomlabs-ai#1331))
([3be2526](headroomlabs-ai@3be2526))
* **proxy:** add versionless Vertex AI routes for Claude Code
compatibility
([headroomlabs-ai#1321](headroomlabs-ai#1321))
([bb3e040](headroomlabs-ai@bb3e040))
* **proxy:** bind before eager preload so a hung compressor load can't
block startup
([headroomlabs-ai#1500](headroomlabs-ai#1500))
([d5ac07f](headroomlabs-ai@d5ac07f))
* **proxy:** build SSL contexts for custom CA bundles
([headroomlabs-ai#1134](headroomlabs-ai#1134))
([561ba17](headroomlabs-ai@561ba17))
* **proxy:** forward request-id headers on the streaming path
([headroomlabs-ai#1100](headroomlabs-ai#1100))
([headroomlabs-ai#1258](headroomlabs-ai#1258))
([3d59df7](headroomlabs-ai@3d59df7))
* **proxy:** gate CCR retrieve/compress endpoints to loopback
([headroomlabs-ai#1338](headroomlabs-ai#1338))
([acafb2d](headroomlabs-ai@acafb2d))
* **proxy:** honor force_kompress routing profile
([headroomlabs-ai#996](headroomlabs-ai#996))
([b4682d6](headroomlabs-ai@b4682d6))
* **proxy:** keep large compression results on the critical path
([headroomlabs-ai#296](headroomlabs-ai#296))
([headroomlabs-ai#1352](headroomlabs-ai#1352))
([90734b6](headroomlabs-ai@90734b6))
* **proxy:** offload /v1/compress to the compression executor to stop
blocking the loop
([headroomlabs-ai#1501](headroomlabs-ai#1501))
([27e010e](headroomlabs-ai@27e010e))
* **proxy:** preserve Responses memory continuations with store=false
([headroomlabs-ai#1103](headroomlabs-ai#1103))
([cdfeeac](headroomlabs-ai@cdfeeac))
* **proxy:** queue mid-turn user messages on non-Bedrock streaming path
([headroomlabs-ai#1377](headroomlabs-ai#1377))
([b09f027](headroomlabs-ai@b09f027))
* **proxy:** register interceptor in explicit transforms list when
HEADROOM_INTERCEPT_ENABLED
([headroomlabs-ai#1376](headroomlabs-ai#1376))
([55c700c](headroomlabs-ai@55c700c))
* **proxy:** report real input tokens on streaming message_start
([headroomlabs-ai#1132](headroomlabs-ai#1132))
([headroomlabs-ai#1305](headroomlabs-ai#1305))
([70cc96a](headroomlabs-ai@70cc96a))
* **proxy:** retry upstream 429 with Retry-After on both forwarders
([headroomlabs-ai#1329](headroomlabs-ai#1329))
([90bee89](headroomlabs-ai@90bee89))
* **proxy:** retry upstream 529 overloaded like 429 on both forwarders
([headroomlabs-ai#1495](headroomlabs-ai#1495))
([547b15d](headroomlabs-ai@547b15d))
* **proxy:** stop re-compressing headroom_retrieve output and emitting
unredeemable markers
([headroomlabs-ai#1323](headroomlabs-ai#1323))
([43494ff](headroomlabs-ai@43494ff))
* **proxy:** strip Codex lite header from OpenAI WebSockets
([headroomlabs-ai#1543](headroomlabs-ai#1543))
([5d3803a](headroomlabs-ai@5d3803a))
* **read-lifecycle:** persist STALE Read originals in the CCR store
([headroomlabs-ai#1488](headroomlabs-ai#1488))
([9157173](headroomlabs-ai@9157173))
* recover persistent proxy feature checks and reject non-Copilot
exchange URL
([headroomlabs-ai#1465](headroomlabs-ai#1465))
([16c638b](headroomlabs-ai@16c638b))
* remove agents.md
([headroomlabs-ai#1540](headroomlabs-ai#1540))
([a7d3360](headroomlabs-ai@a7d3360))
* respect COPILOT_PROVIDER_TYPE env var when provider_type is auto
([headroomlabs-ai#549](headroomlabs-ai#549))
([24cf256](headroomlabs-ai@24cf256))
* restore token-mode compression on frozen prefixes
([headroomlabs-ai#1489](headroomlabs-ai#1489))
([8e0dadf](headroomlabs-ai@8e0dadf))
* **router:** degrade to pure-Python detection on native panic
([headroomlabs-ai#1123](headroomlabs-ai#1123))
([headroomlabs-ai#1260](headroomlabs-ai#1260))
([a00fb67](headroomlabs-ai@a00fb67))
* **rtk:** stop hook registration timing out on a forked daemon
([headroomlabs-ai#1314](headroomlabs-ai#1314))
([9758817](headroomlabs-ai@9758817))
* **smart-crusher:** honor enable_ccr_marker on the opaque-blob path
([headroomlabs-ai#1130](headroomlabs-ai#1130))
([27d6f8e](headroomlabs-ai@27d6f8e))
* **subscription:** only reset 5h contribution on real rollover, not API
jitter
([headroomlabs-ai#1255](headroomlabs-ai#1255))
([8d6c175](headroomlabs-ai@8d6c175))
* **subscription:** run transcript token scan off the event loop
([headroomlabs-ai#1263](headroomlabs-ai#1263))
([f03021f](headroomlabs-ai@f03021f))
* surface output reduction without a restart, and explain $0.00 savings
on Python 3.14
([headroomlabs-ai#1296](headroomlabs-ai#1296))
([c30ec4c](headroomlabs-ai@c30ec4c))
* **tests:** reset whole headroom logger subtree so caplog stays
deterministic
([headroomlabs-ai#1117](headroomlabs-ai#1117))
([fda4670](headroomlabs-ai@fda4670))
* **tls:** add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection
([headroomlabs-ai#1308](headroomlabs-ai#1308))
([headroomlabs-ai#1341](headroomlabs-ai#1341))
([52068dd](headroomlabs-ai@52068dd))
* **tokenizers:** price CJK/Kana/Hangul at ~1 token per char in
EstimatingTokenCounter
([headroomlabs-ai#1093](headroomlabs-ai#1093))
([a35fe86](headroomlabs-ai@a35fe86))
* **transforms:** gate tool string output from lossy compression
([headroomlabs-ai#1307](headroomlabs-ai#1307))
([headroomlabs-ai#1387](headroomlabs-ai#1387))
([c6c921a](headroomlabs-ai@c6c921a))
* **websocket:** harden responses websocket origin handling
([headroomlabs-ai#1481](headroomlabs-ai#1481))
([c632023](headroomlabs-ai@c632023))
* **windows:** pin UTF-8 encoding on text-mode subprocess calls
([headroomlabs-ai#1311](headroomlabs-ai#1311))
([d633e81](headroomlabs-ai@d633e81))
* **wrap:** add Copilot unwrap command
([headroomlabs-ai#1251](headroomlabs-ai#1251))
([b4fde0c](headroomlabs-ai@b4fde0c))
* **wrap:** isolate proxy stdio from proxy.log on Windows
([headroomlabs-ai#1191](headroomlabs-ai#1191))
([959ab0d](headroomlabs-ai@959ab0d))
* **wrap:** keep agent savings opt-in
([headroomlabs-ai#1294](headroomlabs-ai#1294))
([b829ceb](headroomlabs-ai@b829ceb))
* **wrap:** show the dashboard URL when the proxy is already running
([headroomlabs-ai#1313](headroomlabs-ai#1313))
([b0146c4](headroomlabs-ai@b0146c4))


### Performance Improvements

* **compression:** take large cold-start contexts off the synchronous
kompress path
([headroomlabs-ai#1171](headroomlabs-ai#1171))
([headroomlabs-ai#1298](headroomlabs-ai#1298))
([6c68ff4](headroomlabs-ai@6c68ff4))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] CodeAwareCompressor corrupts non-ASCII (CJK/emoji) source: tree-sitter byte offsets indexed into Python str → invalid syntax, 0% compression

2 participants