ci: advisory AI failure-investigation workflow (optional, needs ANTHROPIC_API_KEY)#1351
Open
SharedQA wants to merge 8 commits into
Open
ci: advisory AI failure-investigation workflow (optional, needs ANTHROPIC_API_KEY)#1351SharedQA wants to merge 8 commits into
SharedQA wants to merge 8 commits into
Conversation
Split out of constructorfabric#1317 so the ANTHROPIC_API_KEY funding/ownership decision doesn't block the real quality gates. This workflow is advisory only (never a merge gate): on a same-repo gate failure it reads the logs + co-located specs and posts a root-cause comment. Needs an Anthropic API key as the ANTHROPIC_API_KEY Actions secret; until that's set (and funded), this PR stays parked. Signed-off-by: Kenan Salim <kenan.salim@rolos.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughA new GitHub Actions workflow file is added that listens for the completion of specified gate workflows with a failure conclusion. It gates execution to the same repository to prevent fork-based secret exfiltration, conditionally skips when ChangesAI Failure Investigation Workflow
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
cyberantonz
reviewed
Jun 16, 2026
Review feedback on constructorfabric#1351 (cyberantonz): if the secret is not present the action would error. Gate the work on the key: - job-level env HAS_ANTHROPIC_KEY = (secrets.ANTHROPIC_API_KEY != '') - "Collect failing job logs" and "AI investigation" steps run only when the key is set; otherwise a note step logs the skip and the job passes. The workflow is advisory and never a gate, so a missing key must not fail CI. Signed-off-by: Kenan Salim <kenan.salim@rolos.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
.github/workflows/ai-investigate.yml (2)
84-88: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low valueUnpinned action reference conflicts with the repo's hash-pin policy. zizmor flags
anthropics/claude-code-action@v1as unpinned;actions/checkouton Line 50 is correctly pinned to a SHA. The inline rationale (moving tag for security fixes) is reasonable, but to satisfy the blanket policy consider pinning to a commit SHA with the# v1.xcomment and relying on Dependabot/Renovate to bump it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ai-investigate.yml around lines 84 - 88, The anthropics/claude-code-action@v1 reference uses a moving tag instead of a pinned commit SHA, which conflicts with the repository's hash-pin policy for actions. Replace the `@v1` tag with a specific commit SHA (following the pattern used by actions/checkout elsewhere in the workflow) and add an inline comment like # v1.x to document which version the SHA corresponds to. This allows Dependabot or similar tools to manage updates while maintaining the security benefit of pinned references.Source: Linters/SAST tools
72-75: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueLog-collection fallback can fail the step under
pipefail. Withset -euo pipefail, if the fallbackgh run view ... --logexits non-zero, the| tail -800pipeline returns failure andset -eaborts the step, which (via implicitsuccess()) skips the AI step entirely. For an advisory workflow you likely want best-effort collection.🩹 Make log collection best-effort
gh run view "$RUN_ID" --repo "$GITHUB_REPOSITORY" \ --log-failed > /tmp/failure/failed-jobs.log 2>/dev/null || \ gh run view "$RUN_ID" --repo "$GITHUB_REPOSITORY" --log 2>/dev/null \ - | tail -800 > /tmp/failure/failed-jobs.log + | tail -800 > /tmp/failure/failed-jobs.log || \ + echo "log collection failed" > /tmp/failure/failed-jobs.log🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ai-investigate.yml around lines 72 - 75, The log collection fallback starting with the `gh run view` command with `--log-failed` flag can cause the entire step to fail under `pipefail` if the fallback command exits non-zero. Since this is a best-effort log collection for an advisory workflow, make the log collection command non-fatal by adding appropriate error handling (such as appending `|| true`) to the end of the entire fallback sequence so that failure to collect logs does not abort the step and prevent the AI investigation from proceeding.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ai-investigate.yml:
- Around line 36-53: The ANTHROPIC_API_KEY secret could be exposed to Bash
subprocesses when passed to the anthropics/claude-code-action action without
environment scrubbing. Add CLAUDE_CODE_SUBPROCESS_ENV_SCRUB: '1' to the
job-level env section (alongside the existing HAS_ANTHROPIC_KEY environment
variable) to enable best-effort removal of the API key from subprocess
environments spawned by the Bash tool, mitigating the risk of credential
exfiltration from a same-repo compromised branch.
---
Nitpick comments:
In @.github/workflows/ai-investigate.yml:
- Around line 84-88: The anthropics/claude-code-action@v1 reference uses a
moving tag instead of a pinned commit SHA, which conflicts with the repository's
hash-pin policy for actions. Replace the `@v1` tag with a specific commit SHA
(following the pattern used by actions/checkout elsewhere in the workflow) and
add an inline comment like # v1.x to document which version the SHA corresponds
to. This allows Dependabot or similar tools to manage updates while maintaining
the security benefit of pinned references.
- Around line 72-75: The log collection fallback starting with the `gh run view`
command with `--log-failed` flag can cause the entire step to fail under
`pipefail` if the fallback command exits non-zero. Since this is a best-effort
log collection for an advisory workflow, make the log collection command
non-fatal by adding appropriate error handling (such as appending `|| true`) to
the end of the entire fallback sequence so that failure to collect logs does not
abort the step and prevent the AI investigation from proceeding.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f3d2809f-5d72-4288-b9f4-ccb816c457a5
📒 Files selected for processing (1)
.github/workflows/ai-investigate.yml
…on, best-effort logs Address review on constructorfabric#1351: - Set CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1 so claude-code-action's Bash tool can't leak ANTHROPIC_API_KEY to subprocesses (defense-in-depth on the same-repo path; forks are already gated). - SHA-pin anthropics/claude-code-action to the v1 commit per the repo's hash-pin policy (matches actions/checkout), '# v1' for Dependabot bumps. - Make failing-log collection best-effort so a non-zero gh call can't abort this advisory job under 'set -euo pipefail' (which would skip the AI step). Signed-off-by: SharedQA <122366558+SharedQA@users.noreply.github.com>
…ands Public-repo hardening. The agent runs with an API key on CI failures, so a prompt-injection planted in the failing logs/code could abuse a general shell to exfiltrate ANTHROPIC_API_KEY. Replace '--allowedTools Bash' with read tools (Read/Grep/Glob) plus only the commands the prompt actually needs: Bash(git log:*), Bash(gh pr list|view|comment:*), Bash(gh issue create|list:*) No arbitrary shell. Layered with CLAUDE_CODE_SUBPROCESS_ENV_SCRUB + same-repo gate. Signed-off-by: SharedQA <122366558+SharedQA@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #1317 so the key/funding decision doesn't hold up the real gates.
What it is: advisory only — on a same-repo CI gate failure it reads the logs + co-located specs and posts a root-cause comment on the PR. Never a merge gate. Forks can't trigger it (secret-exfiltration guard). Capped at
--max-turns 30/ 15 min.What it needs: an Anthropic API key (console.anthropic.com) as the
ANTHROPIC_API_KEYActions secret (repo or org-level). Billed as metered API usage to whichever Anthropic org owns the key; small since it only runs on failures.Parked as draft until Constructor decides whether to fund/use it. None of the real quality gates depend on this. Can add a skip-guard so it cleanly no-ops when the secret is unset — say the word.
Summary by CodeRabbit