fix(install): close parent log fd in start_detached_agent#1576
Conversation
start_detached_agent() opened the agent log file and passed it to subprocess.Popen as stdout/stderr, then returned the process without closing the parent's copy of the descriptor. The child inherits the fd, but the parent's copy stayed open, so every `headroom install start` leaked one file descriptor and held the log file open against rotation. Under a tight ulimit or in a container, repeated starts can exhaust the fd limit. Close the parent's copy in a try/finally right after Popen returns. The finally also covers the case where Popen itself raises, so a failed spawn can no longer leak the just-opened handle. Adds regression tests asserting the log handle is closed after a normal start and after a Popen failure. Closes headroomlabs-ai#1554
PR governanceThis PR follows the template and is marked ready for human review. |
JerrettDavis
left a comment
There was a problem hiding this comment.
The fd-leak fix itself is correct and the new focused regressions pass locally:
python -m pytest tests/test_install/test_runtime.py::test_start_detached_agent_closes_parent_log_fd tests/test_install/test_runtime.py::test_start_detached_agent_closes_log_fd_when_popen_raises -q
Result: 2 passed.
One process note: this is now largely superseded by the broader runtime/fd handling in #1556, which also covers related Windows PID/SystemError paths. If #1556 lands first, this PR should probably be closed or rebased down to whatever remains unique. I do not see a correctness issue in this isolated patch.
…d-leak # Conflicts: # CHANGELOG.md
|
Thanks for reviewing and actually running the tests — appreciated. Agreed on the overlap: #1556 covers this same One thing worth separating out: #1556 also claims #1544 (the |
…d-leak # Conflicts: # CHANGELOG.md
JerrettDavis
left a comment
There was a problem hiding this comment.
Looks good. Closing the parent's inherited log handle in a finally block handles both successful starts and Popen failures, and the tests assert both cases directly.
Conflict-free sync against upstream/main (brings in the x-headroom-base-url OpenAI-handler fix plus headroomlabs-ai#1564, headroomlabs-ai#1553, headroomlabs-ai#1576). Provider-preserving native routing is unchanged: applyNativeProviderOverrides() only rewrites providers OpenCode already resolved and skips an empty config, keeping native provider/model identity intact. Validated end-to-end with opencode-zen, opencode-go, and openai (subscription) routing through the local Headroom proxy. Opencode reference pricing and historical-cost accounting land in a separate follow-up push.
🤖 I have created a release *beep* *boop* --- <details><summary>0.29.0</summary> ## [0.29.0](v0.28.0...v0.29.0) (2026-07-03) ### Features * **proxy:** add --lossless no-CCR mode with format-native compaction ([#1721](#1721)) ([c75ebde](c75ebde)) * **stats:** surface Codex WS compression counters in /stats summary ([#1680](#1680)) ([2fe19c3](2fe19c3)) * **transforms:** adaptive Otsu KEEP/DROP threshold (+ land relevance split on main) ([#1726](#1726)) ([eea667a](eea667a)) ### Bug Fixes * **bedrock:** fail fast when session-token auth lacks botocore ([#1553](#1553)) ([54cfa36](54cfa36)) * **bedrock:** route ARNs via converse, named AWS profiles, and au. re… ([#1456](#1456)) ([7d87aa2](7d87aa2)) * **ccr:** honor workspace dir for sqlite store ([#1564](#1564)) ([96e1dfe](96e1dfe)) * **claude:** surface Remote Control proxy incompatibility ([#1610](#1610)) ([4bf7f92](4bf7f92)) * **cli:** stop advertising unwired compression tuning env vars in banner ([#1634](#1634)) ([d5bf98d](d5bf98d)) * **codex:** avoid duplicate headroom provider config ([#1431](#1431)) ([ddd4adf](ddd4adf)) * **compression:** reject lossy unmarked tool output in unit router path ([#1479](#1479)) ([de24cd5](de24cd5)) * **cortex-code:** migrate to current Cortex REST API endpoints + add e2e benchmarks ([#1474](#1474)) ([f00ace6](f00ace6)) * **dashboard:** align token savings headline denominator ([#1653](#1653)) ([646e705](646e705)) * **dashboard:** derive per-project setup URL from live origin ([#1511](#1511)) ([e035aef](e035aef)) * **detection:** contain unidiff panic on orphaned +++ target line ([#1548](#1548)) ([e386c09](e386c09)) * **evals:** CJK-aware F1 tokenization + token estimation ([#1527](#1527)) ([99a8540](99a8540)) * **install:** close parent log fd in start_detached_agent ([#1576](#1576)) ([816cb85](816cb85)) * **install:** use Windows-safe PID liveness probe in runtime_status ([#1544](#1544)) ([#1560](#1560)) ([6b227b9](6b227b9)) * **learn:** aggregate verbosity baselines across projects instead of overwriting ([#1288](#1288)) ([27a5468](27a5468)) * **mcp:** show lifetime totals and label rolling session scope in headroom_stats ([#1428](#1428)) ([1c0e152](1c0e152)) * **memory:** cap local embedder CPU thread oversubscription ([#198](#198)) ([#1559](#1559)) ([b84afbf](b84afbf)) * **memory:** singleflight LocalBackend init to stop cold-start races ([#1691](#1691)) ([bec47a1](bec47a1)) * **openclaw:** detect uv-installed headroom binary in ~/.local/bin ([#1459](#1459)) ([adaeb88](adaeb88)) * **opencode:** preserve custom OpenAI gateway paths ([#1596](#1596)) ([c19347c](c19347c)) * **opencode:** route native providers + load transport plugin, fix Serena context ([#1573](#1573)) ([ad0034f](ad0034f)) * preserve anthropic passthrough tool order ([#1427](#1427)) ([a932247](a932247)) * **proxy/auth:** match real Anthropic OAuth token prefix (sk-ant-oat) ([#1672](#1672)) ([8cddf9b](8cddf9b)) * **proxy:** expose persistent savings metrics ([#1647](#1647)) ([5fe4e7b](5fe4e7b)) * **proxy:** fail open when kompress saturation would exhaust pre-upstream budget ([#1430](#1430)) ([15ac650](15ac650)) * **proxy:** handle streaming CCR retrieval ([#1451](#1451)) ([d337e3b](d337e3b)) * **proxy:** include system/tools/sampling in cache key ([#1473](#1473)) ([312129a](312129a)) * **proxy:** preserve Responses passthrough bytes ([#1598](#1598)) ([2a34a82](2a34a82)) * **proxy:** strip Codex lite header on the HTTP /responses path ([#1663](#1663)) ([9fbd47b](9fbd47b)) * **proxy:** wire --compression-max-workers / HEADROOM_COMPRESSION_MAX_WORKERS ([#1632](#1632)) ([814ffa3](814ffa3)) * **savings:** count cache-read tokens in input cost estimate ([#1429](#1429)) ([72ade37](72ade37)) * skip Magika backend on x86 CPUs without AVX2 ([#1162](#1162)) ([64783d8](64783d8)) * **transforms/content-router:** route grep/log output away from HTML extractor ([#1719](#1719)) ([0d18ef2](0d18ef2)) * **transforms:** bound native content detection with a Windows watchdog ([#575](#575)) ([#1563](#1563)) ([95abca3](95abca3)) * Vertex AI support for Claude Code with ANTHROPIC_VERTEX_BASE_URL ([#1393](#1393)) ([cff7247](cff7247)) * **wrap:** detach the shared proxy on Windows so it survives an ungraceful agent close ([#1464](#1464)) ([6cba441](6cba441)) * **wrap:** preserve custom Vertex base URL ([#1477](#1477)) ([75427bb](75427bb)) * **wrap:** remove rtk instructions from Codex AGENTS.md on unwrap ([#1604](#1604)) ([c9d717c](c9d717c)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…bs-ai#1576) ## Description `start_detached_agent()` opens the agent log file and hands it to `subprocess.Popen` as `stdout`/`stderr`, then returns the process **without closing the parent's copy of the file descriptor**. The child inherits the fd and writes to it, but the parent keeps its own copy open forever. The result: every `headroom install start` leaks one file descriptor in the parent, and the leaked handle pins the log file open so it can't be rotated. On a tight `ulimit` or inside a container, repeated starts can walk straight into the fd limit. ```python # headroom/install/runtime.py — before log_file = open(log_file_path, "a", encoding="utf-8", errors="replace") kwargs = {"stdout": log_file, "stderr": log_file, ...} return subprocess.Popen(command, **kwargs) # parent's log_file never closed ``` The fix closes the parent's copy in a `try/finally` right after `Popen` returns: ```python try: proc = subprocess.Popen(command, **kwargs) finally: # The child has inherited the log file descriptor, so the parent's # copy is dead weight. Closing it (even when Popen raises) avoids # leaking one fd per `headroom install start` and lets the log file # be rotated. log_file.close() return proc ``` The `finally` is deliberate: it also covers the case where `Popen` itself raises (bad executable, fork failure), which would otherwise leak the just-opened handle. This matches the `with open(...)` pattern already used by `run_foreground()` a few lines above. Closes headroomlabs-ai#1554 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/install/runtime.py`: close the parent's log file descriptor in a `try/finally` after `subprocess.Popen` in `start_detached_agent()`, so it is released on the normal path and when `Popen` raises. - `tests/test_install/test_runtime.py`: add two regression tests — one for a normal start, one for `Popen` raising — asserting the parent's log handle is closed afterwards. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output Before the fix (`runtime.py` reverted to its parent commit, new tests kept) — the assertion inspects the *actual* log file handle and finds it still open: ```text E AssertionError: assert False is True E + where False = <_io.TextIOWrapper name='...\deploy\demo\runner.log' mode='a' encoding='utf-8'>.closed FAILED tests/test_install/test_runtime.py::test_start_detached_agent_closes_parent_log_fd FAILED tests/test_install/test_runtime.py::test_start_detached_agent_closes_log_fd_when_popen_raises ============================== 2 failed in 0.43s ============================== ``` After the fix: ```text tests\test_install\test_runtime.py ................ ====================== 16 passed, 1 deselected in 0.35s ======================= ``` (The one deselected test, `test_runtime_start_lock_blocks_another_process`, is a pre-existing failure on my Windows box — it fails identically on a clean checkout of `main` and is unrelated to this change.) ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`). - Exact command / steps: ran the two regression tests, which drive the real `start_detached_agent` code path — real `open()`, real `Popen(stdout=...)`, real (or missing) `close()`. Only `subprocess.Popen` is stubbed so the test never launches an actual detached agent; the fd-lifecycle bug lives entirely in how the parent handles its own handle, and that runs for real. - Observed result: the log file handle the parent passed to `Popen` is `.closed == False` before the fix and `.closed == True` after — for both the normal path and the `Popen`-raises path (output above). - Not tested: I intentionally did not spin up many real detached agents to watch the OS fd table grow — on Windows that means flashing console windows and isn't a clean signal anyway. The handle-state assertion on the real file object is the deterministic equivalent. Did not run the full `mypy headroom` pass (one-line lifecycle change, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Single logical change, no new dependencies, default behavior otherwise unchanged. - Rebased/merged latest `main` to clear a `CHANGELOG.md` conflict. - @chopratejas this is a sibling of headroomlabs-ai#1555 (the `wrap.py` readiness-loop handle leak you've got filed). I scoped this PR to headroomlabs-ai#1554 only; happy to follow up on headroomlabs-ai#1555 separately if you'd like. Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
🤖 I have created a release *beep* *boop* --- <details><summary>0.29.0</summary> ## [0.29.0](headroomlabs-ai/headroom@v0.28.0...v0.29.0) (2026-07-03) ### Features * **proxy:** add --lossless no-CCR mode with format-native compaction ([headroomlabs-ai#1721](headroomlabs-ai#1721)) ([c75ebde](headroomlabs-ai@c75ebde)) * **stats:** surface Codex WS compression counters in /stats summary ([headroomlabs-ai#1680](headroomlabs-ai#1680)) ([2fe19c3](headroomlabs-ai@2fe19c3)) * **transforms:** adaptive Otsu KEEP/DROP threshold (+ land relevance split on main) ([headroomlabs-ai#1726](headroomlabs-ai#1726)) ([eea667a](headroomlabs-ai@eea667a)) ### Bug Fixes * **bedrock:** fail fast when session-token auth lacks botocore ([headroomlabs-ai#1553](headroomlabs-ai#1553)) ([54cfa36](headroomlabs-ai@54cfa36)) * **bedrock:** route ARNs via converse, named AWS profiles, and au. re… ([headroomlabs-ai#1456](headroomlabs-ai#1456)) ([7d87aa2](headroomlabs-ai@7d87aa2)) * **ccr:** honor workspace dir for sqlite store ([headroomlabs-ai#1564](headroomlabs-ai#1564)) ([96e1dfe](headroomlabs-ai@96e1dfe)) * **claude:** surface Remote Control proxy incompatibility ([headroomlabs-ai#1610](headroomlabs-ai#1610)) ([4bf7f92](headroomlabs-ai@4bf7f92)) * **cli:** stop advertising unwired compression tuning env vars in banner ([headroomlabs-ai#1634](headroomlabs-ai#1634)) ([d5bf98d](headroomlabs-ai@d5bf98d)) * **codex:** avoid duplicate headroom provider config ([headroomlabs-ai#1431](headroomlabs-ai#1431)) ([ddd4adf](headroomlabs-ai@ddd4adf)) * **compression:** reject lossy unmarked tool output in unit router path ([headroomlabs-ai#1479](headroomlabs-ai#1479)) ([de24cd5](headroomlabs-ai@de24cd5)) * **cortex-code:** migrate to current Cortex REST API endpoints + add e2e benchmarks ([headroomlabs-ai#1474](headroomlabs-ai#1474)) ([f00ace6](headroomlabs-ai@f00ace6)) * **dashboard:** align token savings headline denominator ([headroomlabs-ai#1653](headroomlabs-ai#1653)) ([646e705](headroomlabs-ai@646e705)) * **dashboard:** derive per-project setup URL from live origin ([headroomlabs-ai#1511](headroomlabs-ai#1511)) ([e035aef](headroomlabs-ai@e035aef)) * **detection:** contain unidiff panic on orphaned +++ target line ([headroomlabs-ai#1548](headroomlabs-ai#1548)) ([e386c09](headroomlabs-ai@e386c09)) * **evals:** CJK-aware F1 tokenization + token estimation ([headroomlabs-ai#1527](headroomlabs-ai#1527)) ([99a8540](headroomlabs-ai@99a8540)) * **install:** close parent log fd in start_detached_agent ([headroomlabs-ai#1576](headroomlabs-ai#1576)) ([816cb85](headroomlabs-ai@816cb85)) * **install:** use Windows-safe PID liveness probe in runtime_status ([headroomlabs-ai#1544](headroomlabs-ai#1544)) ([headroomlabs-ai#1560](headroomlabs-ai#1560)) ([6b227b9](headroomlabs-ai@6b227b9)) * **learn:** aggregate verbosity baselines across projects instead of overwriting ([headroomlabs-ai#1288](headroomlabs-ai#1288)) ([27a5468](headroomlabs-ai@27a5468)) * **mcp:** show lifetime totals and label rolling session scope in headroom_stats ([headroomlabs-ai#1428](headroomlabs-ai#1428)) ([1c0e152](headroomlabs-ai@1c0e152)) * **memory:** cap local embedder CPU thread oversubscription ([headroomlabs-ai#198](headroomlabs-ai#198)) ([headroomlabs-ai#1559](headroomlabs-ai#1559)) ([b84afbf](headroomlabs-ai@b84afbf)) * **memory:** singleflight LocalBackend init to stop cold-start races ([headroomlabs-ai#1691](headroomlabs-ai#1691)) ([bec47a1](headroomlabs-ai@bec47a1)) * **openclaw:** detect uv-installed headroom binary in ~/.local/bin ([headroomlabs-ai#1459](headroomlabs-ai#1459)) ([adaeb88](headroomlabs-ai@adaeb88)) * **opencode:** preserve custom OpenAI gateway paths ([headroomlabs-ai#1596](headroomlabs-ai#1596)) ([c19347c](headroomlabs-ai@c19347c)) * **opencode:** route native providers + load transport plugin, fix Serena context ([headroomlabs-ai#1573](headroomlabs-ai#1573)) ([ad0034f](headroomlabs-ai@ad0034f)) * preserve anthropic passthrough tool order ([headroomlabs-ai#1427](headroomlabs-ai#1427)) ([a932247](headroomlabs-ai@a932247)) * **proxy/auth:** match real Anthropic OAuth token prefix (sk-ant-oat) ([headroomlabs-ai#1672](headroomlabs-ai#1672)) ([8cddf9b](headroomlabs-ai@8cddf9b)) * **proxy:** expose persistent savings metrics ([headroomlabs-ai#1647](headroomlabs-ai#1647)) ([5fe4e7b](headroomlabs-ai@5fe4e7b)) * **proxy:** fail open when kompress saturation would exhaust pre-upstream budget ([headroomlabs-ai#1430](headroomlabs-ai#1430)) ([15ac650](headroomlabs-ai@15ac650)) * **proxy:** handle streaming CCR retrieval ([headroomlabs-ai#1451](headroomlabs-ai#1451)) ([d337e3b](headroomlabs-ai@d337e3b)) * **proxy:** include system/tools/sampling in cache key ([headroomlabs-ai#1473](headroomlabs-ai#1473)) ([312129a](headroomlabs-ai@312129a)) * **proxy:** preserve Responses passthrough bytes ([headroomlabs-ai#1598](headroomlabs-ai#1598)) ([2a34a82](headroomlabs-ai@2a34a82)) * **proxy:** strip Codex lite header on the HTTP /responses path ([headroomlabs-ai#1663](headroomlabs-ai#1663)) ([9fbd47b](headroomlabs-ai@9fbd47b)) * **proxy:** wire --compression-max-workers / HEADROOM_COMPRESSION_MAX_WORKERS ([headroomlabs-ai#1632](headroomlabs-ai#1632)) ([814ffa3](headroomlabs-ai@814ffa3)) * **savings:** count cache-read tokens in input cost estimate ([headroomlabs-ai#1429](headroomlabs-ai#1429)) ([72ade37](headroomlabs-ai@72ade37)) * skip Magika backend on x86 CPUs without AVX2 ([headroomlabs-ai#1162](headroomlabs-ai#1162)) ([64783d8](headroomlabs-ai@64783d8)) * **transforms/content-router:** route grep/log output away from HTML extractor ([headroomlabs-ai#1719](headroomlabs-ai#1719)) ([0d18ef2](headroomlabs-ai@0d18ef2)) * **transforms:** bound native content detection with a Windows watchdog ([headroomlabs-ai#575](headroomlabs-ai#575)) ([headroomlabs-ai#1563](headroomlabs-ai#1563)) ([95abca3](headroomlabs-ai@95abca3)) * Vertex AI support for Claude Code with ANTHROPIC_VERTEX_BASE_URL ([headroomlabs-ai#1393](headroomlabs-ai#1393)) ([cff7247](headroomlabs-ai@cff7247)) * **wrap:** detach the shared proxy on Windows so it survives an ungraceful agent close ([headroomlabs-ai#1464](headroomlabs-ai#1464)) ([6cba441](headroomlabs-ai@6cba441)) * **wrap:** preserve custom Vertex base URL ([headroomlabs-ai#1477](headroomlabs-ai#1477)) ([75427bb](headroomlabs-ai@75427bb)) * **wrap:** remove rtk instructions from Codex AGENTS.md on unwrap ([headroomlabs-ai#1604](headroomlabs-ai#1604)) ([c9d717c](headroomlabs-ai@c9d717c)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Description
start_detached_agent()opens the agent log file and hands it tosubprocess.Popenasstdout/stderr, then returns the process withoutclosing the parent's copy of the file descriptor. The child inherits the fd
and writes to it, but the parent keeps its own copy open forever.
The result: every
headroom install startleaks one file descriptor in theparent, and the leaked handle pins the log file open so it can't be rotated.
On a tight
ulimitor inside a container, repeated starts can walk straightinto the fd limit.
The fix closes the parent's copy in a
try/finallyright afterPopenreturns:The
finallyis deliberate: it also covers the case wherePopenitselfraises (bad executable, fork failure), which would otherwise leak the
just-opened handle. This matches the
with open(...)pattern already used byrun_foreground()a few lines above.Closes #1554
Type of Change
Changes Made
headroom/install/runtime.py: close the parent's log file descriptor in atry/finallyaftersubprocess.Popeninstart_detached_agent(), so it is released on the normal path and whenPopenraises.tests/test_install/test_runtime.py: add two regression tests — one for a normal start, one forPopenraising — asserting the parent's log handle is closed afterwards.CHANGELOG.md: Bug Fixes entry under Unreleased.Testing
pytest)ruff check .)mypy headroom)Test Output
Before the fix (
runtime.pyreverted to its parent commit, new tests kept) —the assertion inspects the actual log file handle and finds it still open:
After the fix:
(The one deselected test,
test_runtime_start_lock_blocks_another_process, is apre-existing failure on my Windows box — it fails identically on a clean
checkout of
mainand is unrelated to this change.)Real Behavior Proof
uv sync --extra dev).start_detached_agentcode path — realopen(), realPopen(stdout=...), real (or missing)close(). Onlysubprocess.Popenis stubbed so the test never launches an actual detached agent; the fd-lifecycle bug lives entirely in how the parent handles its own handle, and that runs for real.Popenis.closed == Falsebefore the fix and.closed == Trueafter — for both the normal path and thePopen-raises path (output above).mypy headroompass (one-line lifecycle change, no new types).Review Readiness
Checklist
Additional Notes
mainto clear aCHANGELOG.mdconflict.wrap.pyreadiness-loop handle leak you've got filed). I scoped this PR to [BUG] File descriptor leak in start_detached_agent — parent never closes log_file after Popen #1554 only; happy to follow up on [BUG] File handle leak in _start_proxy_subprocess if _check_proxy raises during readiness loop #1555 separately if you'd like.