fix: preserve anthropic passthrough tool order (headroomlabs-ai#1427)

aivinay · web-flow · commit a9322477e33e · 2026-06-30T08:38:51.000-05:00
## Description Preserves Anthropic `tools` order when Headroom is forwarding a passthrough/no-optimize request. This fixes a Claude Code style `tool_result` continuation failure against stricter Anthropic-compatible upstreams that treat the client's original tool ordering as part of the conversation state. Closes headroomlabs-ai#1417 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Preserve client-provided Anthropic `tools` order when `optimize=False` or the request is explicitly in Headroom passthrough/bypass mode. - Keep deterministic tool sorting for optimized requests where Headroom may rewrite the body for cache stability. - Avoid sorting batch-request tools before the no-optimize passthrough branch. - Add regression coverage for the Anthropic HTTP path to prove no-optimize forwarding keeps `Read`, then `Bash` tool order. - Update existing cache-stability and byte-faithful forwarding tests so no-optimize/passthrough expects preserved client order while optimized mode still proves deterministic sorting. ## Testing - [x] Focused unit tests pass (`pytest` on touched proxy test files) - [x] Linting passes (`ruff check` and `ruff format --check` on touched files) - [x] Type checking passes (`mypy headroom`) - [x] New regression tests added - [x] Manual testing performed ### Test Output ```text $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with pytest --with pytest-asyncio --with anyio --with 'httpx[http2]' --with fastapi --with pydantic --with tiktoken --with click --with rich --with opentelemetry-api --with opentelemetry-sdk --with zstandard --with openai --with mcp --with uvicorn pytest tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py -q ============================= test session starts ============================== platform darwin -- Python 3.13.11, pytest-9.1.1, pluggy-1.6.0 rootdir: /Users/vinaygupta/Desktop/git/headroom-fix-anthropic-tool-order configfile: pyproject.toml plugins: anyio-4.14.1, asyncio-1.4.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 87 items tests/test_proxy_handler_helpers.py .......................... [ 29%] tests/test_anthropic_stage_timings.py .... [ 34%] tests/test_proxy_anthropic_cache_stability.py ......................... [ 63%] tests/test_proxy_byte_faithful_forwarding.py ........................... [ 94%] ..... [100%] =============================== warnings summary =============================== .../fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. ======================== 87 passed, 1 warning in 5.13s ========================= $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py 5 files already formatted $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.11, local fake Anthropic-compatible upstream, local Headroom proxy launched with `--no-optimize --no-cache --no-rate-limit --stateless`. - Exact command / steps: ran a local reproduction harness that starts a fake `/v1/messages` upstream and Headroom proxy, then sends a Claude Code style two-turn flow: first assistant `Bash` `tool_use`, then user `tool_result`. - Observed result: after this patch, both direct and proxied flows returned `200` for `first_tool_use` and `second_tool_result`. The fake upstream log showed the proxied `tools` array remained `["Read", "Bash"]` on both turns. ```text DIRECT first_tool_use: 200 second_tool_result: 200 PROXIED first_tool_use: 200 second_tool_result: 200 UPSTREAM REQUEST LOG proxied first turn tools: ["Read", "Bash"] proxied tool_result turn tools: ["Read", "Bash"] ``` - Not tested: full `pytest`, full-repo `ruff check .`, `mypy headroom`, or a live third-party Anthropic-compatible provider. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - This PR intentionally does not add documentation because it fixes passthrough behavior rather than introducing a new user-facing option. - The code-comment checklist item is left unchecked because the change is covered by a small helper docstring and regression tests; no extra inline comments seemed necessary. - `CHANGELOG.md` is left unchanged because this is a narrowly scoped bug fix. - Local pytest collection for these proxy tests required a local `headroom._core` extension symlink, which was removed before committing.
diff --git a/headroom/proxy/handlers/anthropic.py b/headroom/proxy/handlers/anthropic.py
@@ -149,6 +149,18 @@ def _sort_tools_deterministically(
             return tools
         return sorted(tools, key=cls._tool_sort_key)
 
+    @classmethod
+    def _tools_for_forwarding(
+        cls,
+        tools: list[dict[str, Any]] | None,
+        *,
+        preserve_order: bool,
+    ) -> list[dict[str, Any]] | None:
+        """Return upstream tools, preserving client order for passthrough requests."""
+        if preserve_order:
+            return tools
+        return cls._sort_tools_deterministically(tools)
+
     @staticmethod
     def _compress_latest_user_turn_images_cache_safe(
         messages: list[dict[str, Any]],
@@ -663,6 +675,7 @@ async def _finalize_pre_upstream() -> None:
                 request.headers.get("x-headroom-bypass", "").lower() == "true"
                 or request.headers.get("x-headroom-mode", "").lower() == "passthrough"
             )
+            preserve_tool_order = _bypass or not self.config.optimize
             if _bypass:
                 logger.info(f"[{request_id}] Bypass: skipping compression (header)")
 
@@ -1843,9 +1856,12 @@ class _DeferredCompressionResult:
             # Update body
             body["messages"] = optimized_messages
             if tools or _original_tools is not None:
-                sorted_tools = self._sort_tools_deterministically(tools)
-                if sorted_tools != tools:
-                    tools = sorted_tools
+                forwarded_tools = self._tools_for_forwarding(
+                    tools,
+                    preserve_order=preserve_tool_order,
+                )
+                if forwarded_tools != tools:
+                    tools = forwarded_tools
                 if tools != _original_tools:
                     body["tools"] = tools
 
@@ -1865,11 +1881,10 @@ class _DeferredCompressionResult:
                 optimized_messages = presend_event.messages
                 body["messages"] = optimized_messages
             if presend_event.tools is not None:
-                sorted_tools = self._sort_tools_deterministically(presend_event.tools)
-                if sorted_tools != presend_event.tools:
-                    tools = sorted_tools
-                else:
-                    tools = presend_event.tools
+                tools = self._tools_for_forwarding(
+                    presend_event.tools,
+                    preserve_order=preserve_tool_order,
+                )
                 if tools or body.get("tools") is not None:
                     if tools != body.get("tools"):
                         body["tools"] = tools
@@ -2897,10 +2912,6 @@ async def handle_anthropic_batch_create(
             params = batch_req.get("params", {})
             canonical_params = dict(params)
             original_tools = canonical_params.get("tools")
-            if original_tools is not None:
-                sorted_tools = self._sort_tools_deterministically(original_tools)
-                if sorted_tools != original_tools:
-                    canonical_params["tools"] = sorted_tools
             messages = params.get("messages", [])
             original_messages = copy.deepcopy(messages)
             model = params.get("model", "unknown")
@@ -2915,6 +2926,11 @@ async def handle_anthropic_batch_create(
                 )
                 continue
 
+            if original_tools is not None:
+                sorted_tools = self._sort_tools_deterministically(original_tools)
+                if sorted_tools != original_tools:
+                    canonical_params["tools"] = sorted_tools
+
             # Apply optimization
             original_tokens = 0  # Initialize before try to prevent UnboundLocalError
             try:
diff --git a/tests/test_anthropic_stage_timings.py b/tests/test_anthropic_stage_timings.py
@@ -245,6 +245,42 @@ def test_anthropic_http_happy_path_emits_stage_timings(stage_log_capture):
     assert "total_pre_upstream" in emitted
 
 
+def test_anthropic_no_optimize_preserves_client_tool_order():
+    tools = [
+        {
+            "name": "Read",
+            "description": "Read a file",
+            "input_schema": {"type": "object", "properties": {}},
+        },
+        {
+            "name": "Bash",
+            "description": "Run a shell command",
+            "input_schema": {"type": "object", "properties": {}},
+        },
+    ]
+    request = _build_request(
+        {
+            "model": "claude-3-5-sonnet-latest",
+            "messages": [{"role": "user", "content": "use a tool"}],
+            "tools": tools,
+        },
+        {"authorization": "Bearer sk-ant-api-test"},
+    )
+    handler = _DummyAnthropicHandler()
+
+    import headroom.tokenizers as _tk
+
+    orig_get = _tk.get_tokenizer
+    _tk.get_tokenizer = lambda model: _DummyTokenizer()
+    try:
+        anyio.run(handler.handle_anthropic_messages, request)
+    finally:
+        _tk.get_tokenizer = orig_get
+
+    _, _, _, forwarded_body = handler.captured
+    assert [tool["name"] for tool in forwarded_body["tools"]] == ["Read", "Bash"]
+
+
 def test_anthropic_http_invalid_body_still_emits_stage_timings(stage_log_capture):
     async def receive():
         # Invalid JSON — produces ``ValueError`` from ``_read_request_json``.
diff --git a/tests/test_proxy_anthropic_cache_stability.py b/tests/test_proxy_anthropic_cache_stability.py
@@ -86,10 +86,32 @@ def _make_proxy_client() -> TestClient:
     return TestClient(app)
 
 
-def test_anthropic_tools_sorted_deterministically_before_forward() -> None:
+@pytest.mark.parametrize(
+    ("optimize", "expected_names"),
+    [
+        (False, ["zeta", "alpha", "mu"]),
+        (True, ["alpha", "mu", "zeta"]),
+    ],
+)
+def test_anthropic_tools_forwarding_order_matches_optimization_mode(
+    optimize: bool,
+    expected_names: list[str],
+) -> None:
     captured = {}
     with _make_proxy_client() as client:
         proxy = client.app.state.proxy
+        proxy.config.optimize = optimize
+        proxy.config.mode = "token"
+
+        if optimize:
+            proxy.anthropic_pipeline.apply = lambda **kwargs: SimpleNamespace(
+                messages=kwargs["messages"],
+                transforms_applied=[],
+                timing={},
+                tokens_before=100,
+                tokens_after=100,
+                waste_signals=None,
+            )
 
         async def _fake_retry(method, url, headers, body, stream=False, **kwargs):  # noqa: ANN001
             captured["body"] = body
@@ -128,7 +150,7 @@ async def _fake_retry(method, url, headers, body, stream=False, **kwargs):  # no
 
         assert response.status_code == 200
         sent_tools = captured["body"]["tools"]
-        assert [t["name"] for t in sent_tools] == ["alpha", "mu", "zeta"]
+        assert [t["name"] for t in sent_tools] == expected_names
 
 
 def test_image_compression_only_applies_to_latest_non_frozen_user_turn() -> None:
@@ -186,10 +208,20 @@ def test_image_compression_does_not_touch_previous_turns_if_last_message_not_use
     assert result[0]["content"][0]["source"]["data"] == "OLD_IMAGE_BYTES"
 
 
-def test_anthropic_batch_tools_sorted_deterministically_before_forward() -> None:
+@pytest.mark.parametrize(
+    ("optimize", "expected_names"),
+    [
+        (False, ["zeta", "alpha", "mu"]),
+        (True, ["alpha", "mu", "zeta"]),
+    ],
+)
+def test_anthropic_batch_tools_forwarding_order_matches_optimization_mode(
+    optimize: bool,
+    expected_names: list[str],
+) -> None:
     captured = {}
     config = ProxyConfig(
-        optimize=False,
+        optimize=optimize,
         cache_enabled=False,
         rate_limit_enabled=False,
         cost_tracking_enabled=False,
@@ -203,6 +235,17 @@ def test_anthropic_batch_tools_sorted_deterministically_before_forward() -> None
 
     with TestClient(app) as client:
         proxy = client.app.state.proxy
+        proxy.config.mode = "token"
+
+        if optimize:
+            proxy.anthropic_pipeline.apply = lambda **kwargs: SimpleNamespace(
+                messages=kwargs["messages"],
+                transforms_applied=[],
+                timing={},
+                tokens_before=100,
+                tokens_after=100,
+                waste_signals=None,
+            )
 
         async def _fake_retry(method, url, headers, body, stream=False, **kwargs):  # noqa: ANN001
             captured["body"] = body
@@ -259,7 +302,7 @@ async def _fake_retry(method, url, headers, body, stream=False, **kwargs):  # no
 
         assert response.status_code == 200
         sent_tools = captured["body"]["requests"][0]["params"]["tools"]
-        assert [t["name"] for t in sent_tools] == ["alpha", "mu", "zeta"]
+        assert [t["name"] for t in sent_tools] == expected_names
 
 
 def test_append_context_targets_latest_non_frozen_user_turn() -> None:
diff --git a/tests/test_proxy_byte_faithful_forwarding.py b/tests/test_proxy_byte_faithful_forwarding.py
@@ -308,10 +308,10 @@ def on_pipeline_event(self, event):  # noqa: ANN001
         return None
 
 
-def _make_no_optimize_app() -> tuple[TestClient, _CapturingTransport]:
-    """Boot a proxy with all transforms disabled and a capturing transport."""
+def _make_anthropic_app(*, optimize: bool) -> tuple[TestClient, _CapturingTransport]:
+    """Boot an Anthropic proxy with a capturing transport."""
     config = ProxyConfig(
-        optimize=False,
+        optimize=optimize,
         cache_enabled=False,
         rate_limit_enabled=False,
         cost_tracking_enabled=False,
@@ -335,6 +335,11 @@ def _make_no_optimize_app() -> tuple[TestClient, _CapturingTransport]:
     return TestClient(app), transport
 
 
+def _make_no_optimize_app() -> tuple[TestClient, _CapturingTransport]:
+    """Boot a proxy with all transforms disabled and a capturing transport."""
+    return _make_anthropic_app(optimize=False)
+
+
 def test_passthrough_no_mutation_byte_equal_sha256() -> None:
     """No transform → upstream SHA-256 equals client-sent SHA-256."""
     client, transport = _make_no_optimize_app()
@@ -448,7 +453,7 @@ def test_anthropic_tools_canonical_order_preserves_byte_faithful_request() -> No
     )
 
 
-def test_anthropic_tools_unsorted_reordered_and_canonicalized() -> None:
+def test_anthropic_tools_unsorted_order_preserves_byte_faithful_request() -> None:
     client, transport = _make_no_optimize_app()
     inbound_dict = {
         "model": "claude-sonnet-4-6",
@@ -459,6 +464,49 @@ def test_anthropic_tools_unsorted_reordered_and_canonicalized() -> None:
             {"name": "alpha"},
         ],
     }
+    inbound_bytes = serialize_body_canonical(inbound_dict)
+
+    response = client.post(
+        "/v1/messages",
+        headers={
+            "x-api-key": "test-key",
+            "anthropic-version": "2023-06-01",
+            "content-type": "application/json",
+        },
+        content=inbound_bytes,
+    )
+    assert response.status_code == 200, response.text
+    upstream = transport.captured_body or b""
+    assert upstream == inbound_bytes
+    forwarded = json.loads(upstream.decode("utf-8"))
+    assert [tool["name"] for tool in forwarded["tools"]] == ["zeta", "alpha"]
+
+
+def test_anthropic_tools_unsorted_reordered_and_canonicalized_when_optimized() -> None:
+    client, transport = _make_anthropic_app(optimize=True)
+    proxy = client.app.state.proxy
+    proxy.config.mode = "token"
+
+    def _fake_apply(**kwargs):
+        return SimpleNamespace(
+            messages=kwargs["messages"],
+            transforms_applied=[],
+            timing={},
+            tokens_before=100,
+            tokens_after=100,
+            waste_signals=None,
+        )
+
+    proxy.anthropic_pipeline.apply = _fake_apply
+    inbound_dict = {
+        "model": "claude-sonnet-4-6",
+        "max_tokens": 64,
+        "messages": [{"role": "user", "content": "plan test"}],
+        "tools": [
+            {"name": "zeta", "description": "later"},
+            {"name": "alpha"},
+        ],
+    }
     expected_dict = {
         **inbound_dict,
         "tools": [
diff --git a/tests/test_proxy_handler_helpers.py b/tests/test_proxy_handler_helpers.py
@@ -570,6 +570,15 @@ def test_anthropic_tool_sort_and_context_append_helpers() -> None:
         "tool",
     ]
     assert AnthropicHandlerMixin._sort_tools_deterministically(None) is None
+    assert AnthropicHandlerMixin._tools_for_forwarding(tools, preserve_order=True) == tools
+    assert [
+        AnthropicHandlerMixin._tool_sort_key(tool)[0]
+        for tool in AnthropicHandlerMixin._tools_for_forwarding(tools, preserve_order=False) or []
+    ] == [
+        "alpha",
+        "beta",
+        "tool",
+    ]
     assert (
         AnthropicHandlerMixin._append_context_to_latest_non_frozen_user_turn(
             [], "ctx", frozen_message_count=0
