Fix update_category silently no-op'ing on empty-string clears#23
Merged
Conversation
`update_category(term_id, {'description': ''})` returns a 200 with no
error but leaves the category's description unchanged on the live
site. The old value is silently preserved and the caller has no way
to tell the update didn't take effect.
The root cause is WP.com's v1.1 category update endpoint: it treats
a form-urlencoded empty string as "don't update this field" rather
than "clear it." Every other shape I tested during an empirical probe
against a live Jetpack-connected site either no-ops the same way
(JSON empty string, JSON null — some return HTTP 400) or points at a
v1.2 endpoint that doesn't exist (HTTP 404 on every v1.2 variant).
The only form-urlencoded shape that reliably clears a text field is
a single NULL byte; WP's input sanitization turns `\x00` into an
empty string server-side and produces the clear the caller asked for.
Two changes in `update_category`:
1. Empty-string substitution. Before posting, any string field in
the caller's payload whose value is `''` gets rewritten to a
single NULL byte. Integer fields (`parent=0`) are left alone.
This is the only reliable shape empirically — documented in the
code comment with the rationale.
2. Verify-after-write. After the POST, compare every field the
caller asked to change against the API response. If any value
doesn't match the caller's intent, raise `update_no_op` instead
of returning a success-shaped dict. This guards against the
empty-string case stopping working in the future AND catches any
new silent-failure modes WP.com introduces on this endpoint.
Comparison uses the caller's original intent (empty string), not
the substituted wire format (NULL byte), so the clear case
verifies cleanly.
This bug surfaced during the PR m#19 end-to-end test. The restore
path called `update_category(id, {'description': ''})` 24 times to
revert descriptions to their pre-apply empty state. All 24 calls
returned success with 0 errors. None of them actually cleared
anything on the live site — the restore reported a clean
`update_category: 24 ops` despite the site being unchanged. With
this fix the substitution makes the clears land, and the
verify-after-write turns any future silent no-op into a loud failure.
End-to-end verified against a live Jetpack-connected site: create a
category with a description, `update_category(id, {'description':
''})`, refetch — description is now empty. Same adapter path, no
workarounds.
Three new tests cover the substitution, the verify-after-write loud
failure, and the verify-accepts-the-clear happy path. Two existing
tests updated to return matching field values in their POST mocks
(previously the mocks returned just `{'ID': 42}` which wouldn't
satisfy the new verify step).
…gory-empty-clears # Conflicts: # lib/adapters/wpcom_adapter.py # tests/test_wpcom_adapter.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
update_category(term_id, {'description': ''})returns 200 with no error but leaves the category's description unchanged on the live site. The old value is silently preserved and the caller has no way to tell the update didn't take effect.The root cause turns out to be in WP.com's v1.1 category update endpoint: it treats a form-urlencoded empty string as "don't update this field" rather than "clear it." I ran an empirical probe against a live Jetpack-connected test site covering every payload shape I could think of — form vs JSON, v1.1 vs v1.2, single space, empty string, null, NULL byte — and the results are pretty clear:
\x00)invalid_inputSo the only reliable clear-a-text-field shape on WP.com v1.1 is a single NULL byte. WP's input sanitization turns
\x00into an empty string on the server side, producing the clear the caller asked for. It's a hack, but it's the only thing that works.Why this matters
This surfaced during the #19 end-to-end test. The restore path called
update_category(id, {'description': ''})24 times to revert descriptions back to their pre-apply empty state. All 24 calls returned success with 0 errors. None of them actually cleared anything on the live site — the restore reportedupdate_category: 24 ops, errors: 0while the site stayed exactly where the apply run left it. For an undo path with a "nothing is lost" promise, that's the worst kind of failure: silent success.What changed
Two things in
update_category:Empty-string substitution before POSTing. For any string field in the caller's payload whose value is
'', substitute'\x00'on the wire. Integer fields likeparent=0are left alone. The rationale is documented in a comment with a pointer to the empirical probe I ran.Verify-after-write. After the POST, compare every field the caller asked to change against the API response. If any value doesn't match the caller's intent, raise
WpcomApiError('update_no_op', ...)instead of returning a success-shaped dict. This guards against:The comparison uses the caller's original intent (empty string), not the substituted wire format (NULL byte), so the clear case still verifies cleanly.
Tests
Three new tests in
TestUpdateCategoryplus two existing tests updated:New:
test_empty_string_substituted_with_null_byte— asserts the body carriesdescription=%00, notdescription=or an empty valuetest_preserves_non_empty_strings— asserts non-empty values are NOT substituted (regression guard for the substitution logic)test_raises_on_silent_no_op— simulates a POST that returns the old value and assertsupdate_no_opis raisedtest_verify_accepts_empty_clear_when_response_matches— happy path: clear request, response shows empty, verify passesUpdated (existing tests):
test_includes_parent— mock response now echoesdescription: 'new'so verify-after-write is satisfiedtest_explicit_parent_used— mock response now echoesparent: 7All 29 tests pass.
Test plan
python3 -m unittest tests.test_wpcom_adapterpasses all 29 testssheer-rockhopper.jurassic.ninja):update_category(id, {'description': ''})via the fixed adapterScope
Just
update_category. No other adapter methods, no agent markdown, no other files. Can land independently of #19, #21, and #22 — none of them touch this code path.Related
set_post_categoriesfix (different method, same "silent success" class of bug)backup()pagination fix (different method, same end-to-end test)Of note: #19's restore path has a broader version of this problem — it doesn't verify that any of its restore operations actually took effect. This PR closes the
update_categoryleg specifically. The general fix (read-back verification for every restore op, or a distinctPartialRestoreError) is worth landing in #19 itself and I've flagged it in the review there.Did I miss anything?