Skip to content

Recover from a duplicate ALTER PUBLICATION ADD TABLE instead of retrying it forever - #5

Open
DamilShahzad7 wants to merge 1 commit into
PgCache:mainfrom
DamilShahzad7:fix/publication-registration-retry-loop
Open

DamilShahzad7 wants to merge 1 commit into
PgCache:mainfrom
DamilShahzad7:fix/publication-registration-retry-loop

Conversation

@DamilShahzad7

Copy link
Copy Markdown

Summary

publication_relations_add (src/cache/writer/publication.rs) issues ALTER PUBLICATION ... ADD TABLE for each newly-active relation, then updates its local publication_oids bookkeeping once the statement succeeds. If Postgres rejects the statement because a named relation is already a publication member, the function currently returns the error before reaching that update — so publication_oids never learns about the relation, and every later registration attempt against it recomputes the identical added set and reissues the identical failing statement. There's no backoff, so this is a live-lock, not just a failed registration.

I ran into this running the project's own conformance harness: it never completed, stuck repeating the same "relation \"onek\" is already member of publication ..." error indefinitely.

Fix

Catch Postgres's duplicate_object (42710) specifically on this statement and treat it as success: sync publication_oids from the attempted set and continue, instead of propagating the error. Any other error still propagates unchanged. This mirrors the existing SqlState-matching convention already used in this codebase (e.g. src/pg/event_trigger.rs).

Testing

  • cargo check --workspace --locked --features proxy — clean.
  • cargo clippy --workspace --locked --features proxy --all-targets — no new warnings from this change (8 pre-existing, unrelated warnings elsewhere, unchanged).
  • Added tests/publication_registration_test.rs, which reproduces the race by adding the test table to the publication out-of-band before pgcache registers it, then asserts the query still gets cached:
    • Passes with this fix.
    • Fails with the fix reverted, reproducing the exact "already member of publication" error from the live-lock, confirming the test actually exercises the bug.
  • Ran the existing tests that touch publication/CDC behavior (schema_change_test, eviction_test, admin_endpoint_test, source_txn_framing_test) alongside the new test; a couple of unrelated tests were intermittently flaky in my container, and I confirmed each one fails identically on unmodified main, so they're pre-existing environment flakiness, not a regression from this change.

@philip-pgcache

Copy link
Copy Markdown
Contributor

Hi Damil, thanks so much for this! Great catch, and the regression test is really helpful.

We're putting a CLA/contributor agreement in place before we merge outside code, so this will sit for a bit. We'll come back to you as soon as it's ready.

@DamilShahzad7

Copy link
Copy Markdown
Author

Hi Philip,

Happy to contribute.

Sounds good! Let me know when the agreement is ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants