test: add codspeed benchmarks for listener duplicate-packet dedup by bluetoothbot · Pull Request #1744 · python-zeroconf/python-zeroconf

bluetoothbot · 2026-05-20T01:46:07Z

What

Three CodSpeed benchmarks for AsyncListener._process_datagram_at_time, exercising the duplicate-packet suppression branch under different stream shapes.

Why

#1743 replaces the single-slot remembered-last-packet dedup (self.data) with a bounded recency window keyed on payload bytes. Landing benchmarks ahead of that change lets CodSpeed compare before/after on the same PR — so we can demonstrate the alternating-flood improvement does not come at the cost of regressing the steady-state hit or unique-stream paths.

Verified on master:

test_alternating_payloads: 200/200 packets bypass dedup (the bug from Security: Duplicate-packet suppression only compares to the most recent packet, allowing trivial bypass #1724).
test_dedup_hit_same_payload: 1/200 reach parse (steady-state dedup works).

After #1743 lands, CodSpeed should show a large speedup on the alternating case and roughly flat results on the other two.

How

tests/benchmarks/test_listener_dedup.py — three @pytest.mark.asyncio benchmarks following the test_send.py pattern (AsyncZeroconf on loopback, async_close in teardown so the project-wide verify_threads_ended fixture stays green).
_InertListener subclasses AsyncListener and stubs handle_query_or_defer, holding the post-dedup work constant across implementations so any delta is attributable to the dedup branch itself.
Toggles registry.has_entries = True directly instead of round-tripping async_register_service through the event loop — the realistic code path stays live without paying registration cost in setup.
200 packets per benchmark iteration; queries (_FLAGS_QR_QUERY) without QU questions so the dedup branch is eligible.

Testing

SKIP_CYTHON=1 poetry run pytest tests/benchmarks/test_listener_dedup.py --no-cov
# 3 passed
poetry run ruff check tests/benchmarks/test_listener_dedup.py
poetry run ruff format --check tests/benchmarks/test_listener_dedup.py

Behavioural check on master (instrumented handle_query_or_defer count):

alternating: 200/200 packets parse
same payload: 1/200 packets parse

Quality Report

Changes: 1 file changed, 128 insertions(+)

Code scan: clean

Tests: passed (4 PASSED)

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

Adds three benchmarks that pin AsyncListener._process_datagram_at_time under packet streams that exercise the dedup branch differently: - test_dedup_hit_same_payload — steady-state dedup hit (one payload repeated). - test_alternating_payloads — A, B, A, B, ... The single-slot remembered-last-packet dedup misses on every packet, so all 200 flood through DNSIncoming parse. This is the shape from issue python-zeroconf#1724 and the path python-zeroconf#1743 fixes. - test_unique_payloads — 200 distinct payloads, never deduped on either implementation. Pins the miss-path storage cost. Landing on master ahead of python-zeroconf#1743 lets CodSpeed compare before/after on the same PR so we can demonstrate the alternating-flood improvement does not come at the cost of regressing the steady- state hit or unique-stream paths.

codecov · 2026-05-20T01:47:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.76%. Comparing base (0e5e637) to head (1b88ce8).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1744   +/-   ##
=======================================
  Coverage   99.76%   99.76%           
=======================================
  Files          33       33           
  Lines        3455     3455           
  Branches      478      478           
=======================================
  Hits         3447     3447           
  Misses          5        5           
  Partials        3        3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codspeed-hq · 2026-05-20T01:48:01Z

Merging this PR will degrade performance by 8.25%

❌ 1 regressed benchmark
✅ 10 untouched benchmarks
🆕 3 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
❌	`test_expire_no_records_to_expire`	14 µs	15.3 µs	-8.25%
🆕	`test_alternating_payloads`	N/A	1.3 ms	N/A
🆕	`test_dedup_hit_same_payload`	N/A	83.3 µs	N/A
🆕	`test_unique_payloads`	N/A	1.4 ms	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing bluetoothbot:koan/benchmark-listener-dedup (1b88ce8) with master (0e5e637)¹}

No successful run was found on master (7f0c476) during the generation of this report, so 0e5e637 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

bdraco · 2026-05-20T02:17:00Z

@bluetoothbot rr

bluetoothbot · 2026-05-20T02:25:41Z

PR Review — test: add codspeed benchmarks for listener duplicate-packet dedup

Clean benchmark addition. Three benchmarks isolate the dedup branch cleanly: _InertListener subclasses (correctly avoiding the slots-assignment trap), registry.has_entries = True shortcut, query packets so the dedup branch is eligible, prime-then-measure for the steady-state case. Matches the repo's existing test_send.py benchmark pattern. CodSpeed's reported 8.25% regression is on an existing benchmark (this PR only adds new ones), so it's drift unrelated to the diff. Suggestions above are all polish, not blockers — merge-ready.

🟢 Suggestions

1. async_close not in finally — leaks on benchmark failure (`tests/benchmarks/test_listener_dedup.py`, L71-88)

If _run (or anything between construction and the close call) raises, aiozc.async_close() is skipped and the project-wide verify_threads_ended conftest fixture will fail at teardown, masking the original error. Same pattern in all three tests. The existing test_send.py benchmarks have the same shape, so this isn't novel, but a try/finally would make the failure mode point at the actual bug instead of a thread-leak red herring. Optional polish, not a blocker.

aiozc = AsyncZeroconf(interfaces=["127.0.0.1"])
...
@benchmark
def _run() -> None:
    ...

await aiozc.async_close()

2. Frozen `t` may not exercise the same code path on both implementations (`tests/benchmarks/test_listener_dedup.py`, L76-83)

All three benchmarks freeze t = current_time_millis() once per _run invocation, so every packet in the 200-loop reuses the same timestamp. For test_dedup_hit_same_payload this is the explicit intent (stay inside the suppression interval). For test_alternating_payloads and test_unique_payloads it's incidental — fine here because the dedup branch decision in #1743's bounded-window proposal is payload-keyed, not time-keyed within a window. Worth a one-line note in each docstring (or a # t frozen on purpose comment) so a future reader doesn't 'fix' this by moving current_time_millis() inside the loop and silently change what's being measured.

@benchmark
def _run() -> None:
    t = current_time_millis()
    for i in range(_ITERATIONS):
        ...

Checklist

No hardcoded secrets
No unsafe deserialization or eval/exec
Resource cleanup on success path
Resource cleanup on error path — suggestion ＃1
Avoids slots per-instance method assignment
Tests verify observable behavior, not source code text
Benchmark setup outside @benchmark block
Documentation/RFC IPs used (192.0.2.0/24)
Ruff lint + format clean (per PR description)

Summary

Clean benchmark addition. Three benchmarks isolate the dedup branch cleanly: _InertListener subclasses (correctly avoiding the slots-assignment trap), registry.has_entries = True shortcut, query packets so the dedup branch is eligible, prime-then-measure for the steady-state case. Matches the repo's existing test_send.py benchmark pattern. CodSpeed's reported 8.25% regression is on an existing benchmark (this PR only adds new ones), so it's drift unrelated to the diff. Suggestions above are all polish, not blockers — merge-ready.

Automated review by Kōan1b88ce8

bluetoothbot · 2026-05-20T02:26:42Z

Rebase with requested adjustments

Branch koan/benchmark-listener-dedup was rebased onto master and review feedback was applied.

Stats

1 file changed, 128 insertions(+)

Actions performed

Already-solved check: negative (confidence=high, reasoning=No commit on master adds benchmarks under tests/benchmarks/test_listener_dedup.py for listener dedup)
Rebased koan/benchmark-listener-dedup onto upstream/master
Pre-push CI check: previous run passed
Force-pushed koan/benchmark-listener-dedup to origin
CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.

Automated by Kōan

bluetoothbot temporarily deployed to release May 20, 2026 01:51 — with GitHub Actions Inactive

bdraco marked this pull request as ready for review May 20, 2026 02:28

bdraco merged commit 068c3f6 into python-zeroconf:master May 20, 2026
35 of 36 checks passed

bluetoothbot deleted the koan/benchmark-listener-dedup branch May 20, 2026 03:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: add codspeed benchmarks for listener duplicate-packet dedup#1744

test: add codspeed benchmarks for listener duplicate-packet dedup#1744
bdraco merged 1 commit into
python-zeroconf:masterfrom
bluetoothbot:koan/benchmark-listener-dedup

bluetoothbot commented May 20, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 20, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented May 20, 2026

Uh oh!

bdraco commented May 20, 2026

Uh oh!

bluetoothbot commented May 20, 2026

Uh oh!

bluetoothbot commented May 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bluetoothbot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Testing

Quality Report

Uh oh!

codecov Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq Bot commented May 20, 2026

Merging this PR will degrade performance by 8.25%

Performance Changes

Footnotes

Uh oh!

bdraco commented May 20, 2026

Uh oh!

bluetoothbot commented May 20, 2026

PR Review — test: add codspeed benchmarks for listener duplicate-packet dedup

🟢 Suggestions

Checklist

Summary

Uh oh!

bluetoothbot commented May 20, 2026

Rebase with requested adjustments

Stats

CI status

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bluetoothbot commented May 20, 2026 •

edited

Loading

codecov Bot commented May 20, 2026 •

edited

Loading