Summary

is_tree_sitter_available() in headroom/transforms/code_compressor.py reports code-aware compression as available based on an import check alone, without ever constructing a parser or attempting a parse. Separately, in at least some dependency combinations the parser fails to initialize at runtime; when it does, the path is caught and input is routed through the generic text compressor with no warning, even though the availability flag and startup banner still report code-aware as on. The result is that code can be compressed with a lossy text strategy that does not preserve structure, while the reported strategy says otherwise.

Root cause

Two issues compound:

1. The availability probe only verifies that the module imports, not that a parser can actually parse:

# code_compressor.py  (_check_tree_sitter_available / is_tree_sitter_available)
import tree_sitter_language_pack   # import succeeds
# ... returns True. No parser is constructed, no parse is attempted.

2. The parser construction assigns a Language produced by tree_sitter_language_pack.get_language() onto a stock tree_sitter.Parser:

# code_compressor.py  _get_parser()
from tree_sitter import Parser
from tree_sitter_language_pack import get_language
parser = Parser()
parser.language = get_language(language)   # can raise on a Parser/Language ABI mismatch

In the version combination I hit locally (tree_sitter and tree_sitter_language_pack built against different core ABIs), the assignment raises, _get_parser re-raises, and the caller falls back to the text compressor. Because step 1 never exercises a real parse, the availability flag stays true regardless.

Impact

Code blocks are compressed with the generic text strategy, which does not preserve code structure. In local measurement this produced materially worse output for code-heavy payloads than a structure-preserving path would, and it happens silently, so the reported strategy does not match what actually ran.

Reproduce

1. Install with the [code] extra and start the proxy (or call compress on a code-heavy input).
2. Observe is_tree_sitter_available() returns true / banner shows code-aware enabled.
3. Send a source file through compression and inspect the strategy actually applied and the output. The output is text-compressed, not AST-compressed.

(A real-behavior before/after capture can be attached once the direction is agreed.)

Proposed direction

If a fix is welcome, one direction that worked for me:

• Make the availability check do a minimal real parse for one grammar instead of an import-only probe, so a broken parser surfaces as unavailable rather than as a silent downgrade.
• Build the parser through the language-pack's own parser factory (get_parser) rather than assigning a pack Language onto a stock Parser, with a thin adapter where the pack node API differs from the stock node API.
• Log once at WARNING when code-aware is requested but the parser is unavailable, so the downgrade is visible.

Question

Is a fix along these lines welcome as a PR? I have a working version with a node-API adapter and a real-parse availability check, with tests, and can open it against this issue if the direction looks right.