fix(ci): make publish-chart resilient to a stale base commit#1342
Conversation
The publish-chart job both patch-bumps the umbrella `version` and rewrites `ingestion.toolboxImage` in charts/insight/values.yaml from the tree checked out at this run's trigger SHA. When two version-bumping PRs merge back-to-back — especially two that touch src/ingestion/** and so both rebuild the toolbox — the second run's trigger SHA does not yet contain the first run's `chore(release)` commit, so both runs: * compute the SAME next umbrella version, and * rewrite the SAME `ingestion.toolboxImage` line to their own build tag. The first run pushes its release commit; the second run's commit-back is then rejected (non-fast-forward). The retry's `git pull --rebase` hits a conflict in charts/insight/values.yaml — specifically on the `ingestion.toolboxImage` line (the version line auto-merges because both runs wrote the same value) — and aborts -> exit 1. A plain job re-run repeats this forever because `checkout` re-pins the same stale SHA. Secondary symptom: both runs `helm push` the same chart version with different contents, so the chart published to GHCR diverges from what main records as that version. Fix: re-anchor the working tree to the live branch tip (`git fetch` + `git reset --hard origin/$GITHUB_REF_NAME`) before any value is computed, so version-compute, the toolboxImage ref and the commit-back are all consistent with the branch tip — and a re-run recomputes against the refreshed tip instead of replaying the stale SHA. Combined with the existing per-ref `concurrency` serialisation the final push fast-forwards, so the fragile rebase-retry is replaced with a fail-loud guard. Reproduced by run 27535189783 (constructorfabric#1306 Figma then constructorfabric#1305 Workday merged back-to-back, both touching src/ingestion/**). Note: bump-descriptors shares the same rebase-retry pattern; left untouched here to keep this fix scoped to the reproduced failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 31 minutes and 11 seconds. Learn how PR review limits work. To continue reviewing without waiting, enable usage-based billing in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Problem
The
publish-chartjob inbuild-images.ymlfailed onmain(run 27535189783) at the Commit version bumps back to main step:Re-running the job reproduced the failure identically — confirming this is not a transient race but a structural defect.
Root cause
publish-chartboth patch-bumps the umbrellaversionand rewritesingestion.toolboxImageincharts/insight/values.yaml, reading from / writing to the tree checked out at the run's trigger SHA.Two PRs that both touch
src/ingestion/**were merged back-to-back:Each merge triggers an independent run, and because both touch ingestion, both rebuild the toolbox. The second run's trigger SHA (
03af566) does not contain the first run'schore(release)commit, so both runs:0.1.67), andingestion.toolboxImageline to their own build tag.The first run (
28de74f) pushes its release commit; the second run's commit-back is then rejected (non-fast-forward). The retry'sgit pull --rebaseconflicts incharts/insight/values.yaml— specifically on theingestion.toolboxImageline (the version line auto-merges, both sides wrote0.1.67) — and aborts →exit 1. A plain job re-run repeats forever becausecheckoutre-pins the same stale SHA.Secondary symptom: both runs
helm pushchart0.1.67with different contents (different toolbox tags), so the artifact published to GHCR diverges from whatmainrecords as0.1.67.Why other approaches don't fit
concurrencyis already set (build-images-${{ github.ref }},cancel-in-progress: false) and serialises runs — but each run is still pinned to its own trigger SHA, so it doesn't help; if anything it makes the stale-base collision near-certain for close merges.cancel-in-progress: trueis incompatible with this pipeline: change detection is per-push (diff vs parent), so cancelling an intermediate run drops that commit's image builds, and the connector descriptor recursion-break relies on the parent-diff — basing it on the last release instead would cause an infinite rebuild loop.pushtomain, outside the queue — so it doesn't gate the racing release runs.Fix
git fetch+git reset --hard origin/$GITHUB_REF_NAME) at job start, before any value is computed. Version-compute, thetoolboxImageref, and the commit-back are now all consistent with the branch tip, and a re-run recomputes against the refreshed tip instead of replaying a stale SHA.concurrencyserialisation, the final push fast-forwards; the only remaining way the tip moves is a direct human push, where rebasing the bump is unresolvable anyway.Scope note
bump-descriptorsshares the same rebase-retry pattern. Left untouched here to keep this PR scoped to the reproduced failure; can follow up if desired.Testing
python3 -c "import yaml; yaml.safe_load(...)"passes (YAML valid).🤖 Generated with Claude Code