Skip to content

fix(proxy): preserve byte-faithful Anthropic tool forwarding#1222

Merged
chopratejas merged 5 commits into
headroomlabs-ai:mainfrom
rodboev:pr/1042-prefix-cache-byte-faithful
Jun 21, 2026
Merged

fix(proxy): preserve byte-faithful Anthropic tool forwarding#1222
chopratejas merged 5 commits into
headroomlabs-ai:mainfrom
rodboev:pr/1042-prefix-cache-byte-faithful

Conversation

@rodboev

@rodboev rodboev commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Description

Anthropic tool sorting was rewriting tools lists even when canonical order was already present, which forced a mutation path that bypassed byte-faithful forwarding and reduced prefix-cache hit stability on repeated /v1/messages turns.

Closes #1042

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • changed Anthropic tool canonicalization so already-canonical tool arrays are not rewritten in both single-request and batch paths
  • preserved byte-faithful forwarding for no-op single-request canonical tool-order requests by avoiding unnecessary body["tools"] reassignment
  • kept canonicalization behavior intact for out-of-order tool arrays
  • added one true regression proof for the PRE_SEND empty-tools path, plus forward-coverage tests for canonical no-op and real-sort-mutation request bodies in test_proxy_byte_faithful_forwarding.py
  • updated CHANGELOG.md to document the prefix-cache fix

Testing

  • Unit tests pass (uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py -x -v)
  • Linting passes (uv run ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py)
  • Type checking passes (uv run mypy headroom) or explain N/A truthfully
  • New tests added for new functionality when applicable
  • Manual testing performed

Test Output

uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py -x -v
56 passed in 12.42s
uv run ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py
All checks passed!
uv run ruff format headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py --check
3 files already formatted

Real Behavior Proof

  • Environment: Windows, local proxy test environment, Anthropic request-forwarding path
  • Exact command / steps: post /v1/messages requests with canonical tool order and then intentionally unsorted tool order through the TestClient path with the no-optimize app variant
  • Observed result: the PRE_SEND empty-tools runtime path now keeps body_mutated false where base would mark the request mutated, canonical-order tool payloads still preserve exact inbound bytes end-to-end, and unsorted tool-order payloads are still canonicalized as expected.
  • Batch coverage: no dedicated runtime batch regression test was added; batch no-op/mutation correctness is addressed through the same compare-and-assign pattern on both batch canonical-sort call sites.
  • Not tested: full-suite behavior outside the touched Anthropic forwarding regression surface

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the CHANGELOG.md if applicable

Screenshots (if applicable)

Not applicable.

Additional Notes

The important boundary is not whether tool arrays can be sorted. The real contract is whether a no-op canonicalization should count as a mutation. This change keeps canonicalization for real reorder cases and restores byte-faithful forwarding for already-canonical requests.

@github-actions

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: ready for review Pull request body is complete and the author marked it ready for human review label Jun 21, 2026
@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 52.17391% with 11 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
headroom/proxy/handlers/anthropic.py 52.17% 8 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!

@chopratejas chopratejas merged commit 1f18d59 into headroomlabs-ai:main Jun 21, 2026
23 checks passed
@github-actions github-actions Bot mentioned this pull request Jun 22, 2026
chopratejas pushed a commit that referenced this pull request Jun 22, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.27.0</summary>

##
[0.27.0](v0.26.0...v0.27.0)
(2026-06-22)


### Features

* **cli:** add headroom doctor setup diagnostics
([#926](#926))
([e45cf4e](e45cf4e))
* **cli:** add headroom update command and release banner
([#1088](#1088))
([26be2c3](26be2c3))
* compression extraction — Rust knob exposure, CCR hardening, traffic
audits ([#818](#818))
([b7be381](b7be381))
* measure and surface token throughput (tokens/sec) through the proxy
([#983](#983))
([0d89c67](0d89c67))
* output-token reduction — verbosity shaper, per-user learning,
counterfactual savings
([#965](#965))
([a99dc61](a99dc61))
* **policy:** decay P_alive from idle time near cache TTL
([#856](#856) P3b)
([#1028](#1028))
([fe4f9ee](fe4f9ee))
* **providers:** add Cortex Code (Snowflake CoCo) as a supported agent
([#1190](#1190))
([d9d0bf4](d9d0bf4))
* **proxy:** cc-switch reconciler — keep Headroom in the request path
alongside cc-switch
([#1030](#1030))
([e8fc8a0](e8fc8a0))
* **proxy:** hot-reload live env knobs so a reused proxy picks them up
without a restart
([#1090](#1090))
([6904d47](6904d47))
* **proxy:** make COMPRESSION_TIMEOUT_SECONDS configurable via env
([#946](#946))
([#991](#991))
([addebdb](addebdb))
* **transforms:** tabular + spreadsheet (.xlsx/.xls) compression
([#1128](#1128))
([d789a7c](d789a7c))
* **vertex:** turnkey Claude Code + Vertex compression (+ fixes from the
Vertex review)
([#1113](#1113))
([0e05915](0e05915))


### Bug Fixes

* **ccr:** accept 12-char SmartCrusher hashes in tool injection
([#1095](#1095))
([#1141](#1141))
([9f7f3ad](9f7f3ad))
* **ccr:** return stored content when headroom_retrieve query matches
nothing ([#1213](#1213))
([#1236](#1236))
([08fb845](08fb845))
* **content-router:** honor target_ratio in compression cache + add
proxy --target-ratio flag
([#1108](#1108))
([8894ee0](8894ee0))
* **dashboard:** light-mode backgrounds + aligned savings tables
([#1064](#1064))
([5eae32b](5eae32b))
* **deps:** make litellm optional on Python 3.14
([#956](#956))
([#993](#993))
([b2f04e4](b2f04e4))
* **e2e:** align Codex wrap e2e with global-only RTK guidance
([#1240](#1240))
([#1254](#1254))
([bc12ace](bc12ace))
* **init:** set ENABLE_TOOL_SEARCH=true so Claude Code keeps deferring
tools ([#746](#746))
([#995](#995))
([500ec2b](500ec2b))
* **kompress:** never block the request path on the cold-cache model
download ([#1161](#1161))
([3fc2a78](3fc2a78))
* **memory:** use ONNX embedder for `wrap --memory` sync
([#1092](#1092))
([#1262](#1262))
([4f9feda](4f9feda))
* **openclaw:** wrap plugin export as {register} object for OpenClaw
2026.x compatibility
([#1218](#1218))
([2e6c442](2e6c442))
* **providers:** update DeepSeek V3 context limit from 128K to 1M
([#1038](#1038))
([#1137](#1137))
([bcabc5c](bcabc5c))
* **proxy:** allow disabling periodic TOIN stats logging
([#1265](#1265))
([b5f63d8](b5f63d8))
* **proxy:** honor HEADROOM_EXCLUDE_TOOLS for Codex /v1/responses tool
outputs ([#940](#940))
([#1053](#1053))
([f03e77b](f03e77b))
* **proxy:** preserve byte-faithful Anthropic tool forwarding
([#1222](#1222))
([1f18d59](1f18d59))
* **proxy:** route Codex OAuth image requests
([#1215](#1215))
([381d771](381d771))
* **proxy:** scope CORS to loopback + gate operator/content endpoints
([#1226](#1226))
([bd55a42](bd55a42))
* **proxy:** stamp X-Client: codex on Responses endpoint for
unidentified callers
([#1036](#1036))
([b0cd032](b0cd032))
* **proxy:** treat NODE_EXTRA_CA_CERTS as additive, not replacement
([#998](#998))
([#1031](#1031))
([c987283](c987283))
* **telemetry:** switch anonymous telemetry to opt-in (off by default)
([#1223](#1223))
([b998697](b998697))
* **tokenizers:** bound tiktoken vocab load so a stalled download cannot
hang requests
([#956](#956))
([#994](#994))
([7e86baf](7e86baf))
* **unwrap:** remove ANTHROPIC_BASE_URL + ENABLE_TOOL_SEARCH and init
hooks on unwrap
([#992](#992))
([5b84691](5b84691))
* **wrap:** keep Codex RTK guidance global
([#1240](#1240))
([7c26a54](7c26a54))
* **wrap:** percent-encode non-ASCII cwd names in X-Headroom-Project
header ([#1071](#1071))
([9f712cc](9f712cc))
* **wrap:** write env.ANTHROPIC_BASE_URL to settings.json so
daemon-spawned conversations inherit proxy
([#951](#951))
([#1078](#1078))
([a554c3a](a554c3a))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
studyzy pushed a commit to studyzy/headroom that referenced this pull request Jun 24, 2026
…mlabs-ai#1222)

## Description

Anthropic tool sorting was rewriting `tools` lists even when canonical
order was already present, which forced a mutation path that bypassed
byte-faithful forwarding and reduced prefix-cache hit stability on
repeated `/v1/messages` turns.

Closes headroomlabs-ai#1042

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- changed Anthropic tool canonicalization so already-canonical tool
arrays are not rewritten in both single-request and batch paths
- preserved byte-faithful forwarding for no-op single-request canonical
tool-order requests by avoiding unnecessary `body["tools"]` reassignment
- kept canonicalization behavior intact for out-of-order tool arrays
- added one true regression proof for the PRE_SEND empty-tools path,
plus forward-coverage tests for canonical no-op and real-sort-mutation
request bodies in `test_proxy_byte_faithful_forwarding.py`
- updated `CHANGELOG.md` to document the prefix-cache fix

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_proxy_handler_helpers.py
tests/test_proxy_byte_faithful_forwarding.py -x -v`)
- [x] Linting passes (`uv run ruff check
headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
tests/test_proxy_byte_faithful_forwarding.py`)
- [ ] Type checking passes (`uv run mypy headroom`) or explain N/A
truthfully
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed

### Test Output

```text
uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py -x -v
56 passed in 12.42s
uv run ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py
All checks passed!
uv run ruff format headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py --check
3 files already formatted
```

## Real Behavior Proof

- Environment: Windows, local proxy test environment, Anthropic
request-forwarding path
- Exact command / steps: post `/v1/messages` requests with canonical
tool order and then intentionally unsorted tool order through the
`TestClient` path with the no-optimize app variant
- Observed result: the PRE_SEND empty-tools runtime path now keeps
`body_mutated` false where base would mark the request mutated,
canonical-order tool payloads still preserve exact inbound bytes
end-to-end, and unsorted tool-order payloads are still canonicalized as
expected.
- Batch coverage: no dedicated runtime batch regression test was added;
batch no-op/mutation correctness is addressed through the same
compare-and-assign pattern on both batch canonical-sort call sites.
- Not tested: full-suite behavior outside the touched Anthropic
forwarding regression surface

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

Not applicable.

## Additional Notes

The important boundary is not whether tool arrays can be sorted. The
real contract is whether a no-op canonicalization should count as a
mutation. This change keeps canonicalization for real reorder cases and
restores byte-faithful forwarding for already-canonical requests.
studyzy pushed a commit to studyzy/headroom that referenced this pull request Jun 24, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.27.0</summary>

##
[0.27.0](headroomlabs-ai/headroom@v0.26.0...v0.27.0)
(2026-06-22)


### Features

* **cli:** add headroom doctor setup diagnostics
([headroomlabs-ai#926](headroomlabs-ai#926))
([e45cf4e](headroomlabs-ai@e45cf4e))
* **cli:** add headroom update command and release banner
([headroomlabs-ai#1088](headroomlabs-ai#1088))
([26be2c3](headroomlabs-ai@26be2c3))
* compression extraction — Rust knob exposure, CCR hardening, traffic
audits ([headroomlabs-ai#818](headroomlabs-ai#818))
([b7be381](headroomlabs-ai@b7be381))
* measure and surface token throughput (tokens/sec) through the proxy
([headroomlabs-ai#983](headroomlabs-ai#983))
([0d89c67](headroomlabs-ai@0d89c67))
* output-token reduction — verbosity shaper, per-user learning,
counterfactual savings
([headroomlabs-ai#965](headroomlabs-ai#965))
([a99dc61](headroomlabs-ai@a99dc61))
* **policy:** decay P_alive from idle time near cache TTL
([headroomlabs-ai#856](headroomlabs-ai#856) P3b)
([headroomlabs-ai#1028](headroomlabs-ai#1028))
([fe4f9ee](headroomlabs-ai@fe4f9ee))
* **providers:** add Cortex Code (Snowflake CoCo) as a supported agent
([headroomlabs-ai#1190](headroomlabs-ai#1190))
([d9d0bf4](headroomlabs-ai@d9d0bf4))
* **proxy:** cc-switch reconciler — keep Headroom in the request path
alongside cc-switch
([headroomlabs-ai#1030](headroomlabs-ai#1030))
([e8fc8a0](headroomlabs-ai@e8fc8a0))
* **proxy:** hot-reload live env knobs so a reused proxy picks them up
without a restart
([headroomlabs-ai#1090](headroomlabs-ai#1090))
([6904d47](headroomlabs-ai@6904d47))
* **proxy:** make COMPRESSION_TIMEOUT_SECONDS configurable via env
([headroomlabs-ai#946](headroomlabs-ai#946))
([headroomlabs-ai#991](headroomlabs-ai#991))
([addebdb](headroomlabs-ai@addebdb))
* **transforms:** tabular + spreadsheet (.xlsx/.xls) compression
([headroomlabs-ai#1128](headroomlabs-ai#1128))
([d789a7c](headroomlabs-ai@d789a7c))
* **vertex:** turnkey Claude Code + Vertex compression (+ fixes from the
Vertex review)
([headroomlabs-ai#1113](headroomlabs-ai#1113))
([0e05915](headroomlabs-ai@0e05915))


### Bug Fixes

* **ccr:** accept 12-char SmartCrusher hashes in tool injection
([headroomlabs-ai#1095](headroomlabs-ai#1095))
([headroomlabs-ai#1141](headroomlabs-ai#1141))
([9f7f3ad](headroomlabs-ai@9f7f3ad))
* **ccr:** return stored content when headroom_retrieve query matches
nothing ([headroomlabs-ai#1213](headroomlabs-ai#1213))
([headroomlabs-ai#1236](headroomlabs-ai#1236))
([08fb845](headroomlabs-ai@08fb845))
* **content-router:** honor target_ratio in compression cache + add
proxy --target-ratio flag
([headroomlabs-ai#1108](headroomlabs-ai#1108))
([8894ee0](headroomlabs-ai@8894ee0))
* **dashboard:** light-mode backgrounds + aligned savings tables
([headroomlabs-ai#1064](headroomlabs-ai#1064))
([5eae32b](headroomlabs-ai@5eae32b))
* **deps:** make litellm optional on Python 3.14
([headroomlabs-ai#956](headroomlabs-ai#956))
([headroomlabs-ai#993](headroomlabs-ai#993))
([b2f04e4](headroomlabs-ai@b2f04e4))
* **e2e:** align Codex wrap e2e with global-only RTK guidance
([headroomlabs-ai#1240](headroomlabs-ai#1240))
([headroomlabs-ai#1254](headroomlabs-ai#1254))
([bc12ace](headroomlabs-ai@bc12ace))
* **init:** set ENABLE_TOOL_SEARCH=true so Claude Code keeps deferring
tools ([headroomlabs-ai#746](headroomlabs-ai#746))
([headroomlabs-ai#995](headroomlabs-ai#995))
([500ec2b](headroomlabs-ai@500ec2b))
* **kompress:** never block the request path on the cold-cache model
download ([headroomlabs-ai#1161](headroomlabs-ai#1161))
([3fc2a78](headroomlabs-ai@3fc2a78))
* **memory:** use ONNX embedder for `wrap --memory` sync
([headroomlabs-ai#1092](headroomlabs-ai#1092))
([headroomlabs-ai#1262](headroomlabs-ai#1262))
([4f9feda](headroomlabs-ai@4f9feda))
* **openclaw:** wrap plugin export as {register} object for OpenClaw
2026.x compatibility
([headroomlabs-ai#1218](headroomlabs-ai#1218))
([2e6c442](headroomlabs-ai@2e6c442))
* **providers:** update DeepSeek V3 context limit from 128K to 1M
([headroomlabs-ai#1038](headroomlabs-ai#1038))
([headroomlabs-ai#1137](headroomlabs-ai#1137))
([bcabc5c](headroomlabs-ai@bcabc5c))
* **proxy:** allow disabling periodic TOIN stats logging
([headroomlabs-ai#1265](headroomlabs-ai#1265))
([b5f63d8](headroomlabs-ai@b5f63d8))
* **proxy:** honor HEADROOM_EXCLUDE_TOOLS for Codex /v1/responses tool
outputs ([headroomlabs-ai#940](headroomlabs-ai#940))
([headroomlabs-ai#1053](headroomlabs-ai#1053))
([f03e77b](headroomlabs-ai@f03e77b))
* **proxy:** preserve byte-faithful Anthropic tool forwarding
([headroomlabs-ai#1222](headroomlabs-ai#1222))
([1f18d59](headroomlabs-ai@1f18d59))
* **proxy:** route Codex OAuth image requests
([headroomlabs-ai#1215](headroomlabs-ai#1215))
([381d771](headroomlabs-ai@381d771))
* **proxy:** scope CORS to loopback + gate operator/content endpoints
([headroomlabs-ai#1226](headroomlabs-ai#1226))
([bd55a42](headroomlabs-ai@bd55a42))
* **proxy:** stamp X-Client: codex on Responses endpoint for
unidentified callers
([headroomlabs-ai#1036](headroomlabs-ai#1036))
([b0cd032](headroomlabs-ai@b0cd032))
* **proxy:** treat NODE_EXTRA_CA_CERTS as additive, not replacement
([headroomlabs-ai#998](headroomlabs-ai#998))
([headroomlabs-ai#1031](headroomlabs-ai#1031))
([c987283](headroomlabs-ai@c987283))
* **telemetry:** switch anonymous telemetry to opt-in (off by default)
([headroomlabs-ai#1223](headroomlabs-ai#1223))
([b998697](headroomlabs-ai@b998697))
* **tokenizers:** bound tiktoken vocab load so a stalled download cannot
hang requests
([headroomlabs-ai#956](headroomlabs-ai#956))
([headroomlabs-ai#994](headroomlabs-ai#994))
([7e86baf](headroomlabs-ai@7e86baf))
* **unwrap:** remove ANTHROPIC_BASE_URL + ENABLE_TOOL_SEARCH and init
hooks on unwrap
([headroomlabs-ai#992](headroomlabs-ai#992))
([5b84691](headroomlabs-ai@5b84691))
* **wrap:** keep Codex RTK guidance global
([headroomlabs-ai#1240](headroomlabs-ai#1240))
([7c26a54](headroomlabs-ai@7c26a54))
* **wrap:** percent-encode non-ASCII cwd names in X-Headroom-Project
header ([headroomlabs-ai#1071](headroomlabs-ai#1071))
([9f712cc](headroomlabs-ai@9f712cc))
* **wrap:** write env.ANTHROPIC_BASE_URL to settings.json so
daemon-spawned conversations inherit proxy
([headroomlabs-ai#951](headroomlabs-ai#951))
([headroomlabs-ai#1078](headroomlabs-ai#1078))
([a554c3a](headroomlabs-ai@a554c3a))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

2 participants