Skip to content

Scan reopened pull requests instead of passing them unscanned - #460

Draft
David Larsen (dc-larsen) wants to merge 7 commits into
mainfrom
fix/scan-reopened-pull-requests
Draft

David Larsen (dc-larsen) wants to merge 7 commits into
mainfrom
fix/scan-reopened-pull-requests

Conversation

@dc-larsen

@dc-larsen David Larsen (dc-larsen) commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

With --scm github, a pull_request run with action reopened exits 0 without scanning:

Pull Request Action reopened is not a supported type
exit=0

The example workflow (workflows/github-actions.yml) subscribes to opened, synchronize, reopened, and GitHub's default pull_request types are the same three. GitHub doesn't document which run a required check uses when the same job runs more than once on a commit. So after a close and reopen, the passing run that never scanned can satisfy the required check on a commit an earlier run failed. Reported by a customer who blocks merges on --strict-blocking.

Root Cause

Github.check_event_type only accepted opened and synchronize for pull_request. Every other action, and every event name it didn't recognize (pull_request_target, merge_group, schedule, ...), hit sys.exit(0). The action case logged at INFO and the unknown-event case at ERROR, but both exited 0, so the job passed.

Fix

  • reopened and ready_for_review now return "diff" and run the same comparison as opened/synchronize (PULL_REQUEST_SCAN_ACTIONS in socketsecurity/core/scm/github.py). Both can leave a head commit without a current result: reopened brings back a PR whose last run may have failed, and ready_for_review is the first real run in workflows that skip drafts with if: !github.event.pull_request.draft. Matching is now case-insensitive on the action.
  • Skipped actions (closed, labeled, edited, ...) and unhandled events still exit 0. The README exit-code table documents 0 as "skipped SCM event", and turning those into failures would turn every existing closed/labeled run red across current pipelines. Instead each skip now logs a WARNING that says no scan ran, the run exits 0, and on a required check that pass does not reflect a scan. That keeps the behavior change limited to the two actions above.
  • pull_request_target is not treated as pull_request. On that event GITHUB_SHA/GITHUB_REF point at the base branch and the default checkout is the base branch, so a "PR scan" would compare the base against itself and pass. Changed-file detection in git_interface.py also keys on pull_request. It gets its own warning telling users to trigger on pull_request.
  • GitLab: there is no action filter (every merge_request_event pipeline already diffs, so there's no reopened equivalent). Unhandled pipeline sources get the same explicit warning instead of log.error + exit 0. Bitbucket and Azure have no SCM adapter, so they have no event filter to fix. They run through --scm api.
  • A missing action or event name gets its own warning. With no EVENT_ACTION and no action in GITHUB_EVENT_PATH, the run says it could not determine the pull_request action, instead of "action 'None' is not scanned ... remove this action from the workflow". The same applies to an unset GITHUB_EVENT_NAME and, on GitLab, an unset CI_PIPELINE_SOURCE. This only happens with --scm github/gitlab outside their CI. Still exit 0.
  • The skip warnings list issue_comment as handling ignore commands, not as a scanned event (it runs the comment flow, never a scan).
  • Example workflow: every listed action now scans. Added a comment telling users not to add actions the CLI skips.
  • docs/ci-cd.md: a "Skipped events" section (below the scan-type paragraphs) lists what scans and what is skipped, with notes on merge_group (the pull_request run is the gate) and on fork pull requests (pull_request_target is not a substitute; pull_request from a fork gets no secrets and exits 3).
  • A second gap with the same cause: the example workflow runs the same socket-security job on push: branches: ['**']. A new commit on a PR branch triggers both push and pull_request on the same head commit. The pull_request run diffs and can fail; the push run is a full scan and always exits 0 (confirmed below). With one job name, a required check can resolve against the passing push run. A new "Push and pull request runs on the same commit" section in docs/ci-cd.md, and a comment on the example's push trigger, give two fixes: restrict push to the default branch, or run the pull request scan in its own job and require that job's check.

Not changed, for a maintainer call:

  • merge_group (merge queues) is still skipped with exit 0. A required check that runs on merge_group passes without scanning. Scanning it properly needs a base/head pair from the merge-group payload, which is more than this fix.
  • edited can retarget a PR's base branch, but it also fires on every title/description edit, so it stays skipped. The CLI's baseline is the repository head scan, so a retarget doesn't change the comparison anyway.
  • The example workflow keeps push: branches: ['**']. All three example workflows (GitHub, GitLab, Bitbucket) deliberately scan every commit on every branch, and restricting push would stop scanning branches without a pull request. Branch protection behavior for duplicate job names also couldn't be tested here. The docs and the workflow comment explain the overlap instead.

Testing

  • tests/unit/test_github_event_filter.py (new, 25 cases): scanned actions return "diff"; push/workflow_dispatch (with and without a PR number) and issue_comment routing is pinned; closed/labeled/edited, pull_request_target/merge_group/schedule and GitLab schedule exit 0 with a "No scan ran" warning; a missing action, GITHUB_EVENT_NAME or CI_PIPELINE_SOURCE gets the "could not determine" warning; a full cli() run with an existing blocker under --strict-blocking exits 1 and posts the comment for reopened, ready_for_review and synchronize, and exits 0 without scanning for closed. 15 of the 25 fail on main (the 10 that pass are behavior main already has).
  • tests/core/test_cli_exit_contract.py: the skipped-event contract test now also covers pull_request_target and asserts the warning (all 4 cases fail on main).
  • Full suite with API key variables unset: uv run pytest: 1027 passed (1001 on main). ruff check socketsecurity tests, uv lock --locked --offline, git diff --check, and uv build + twine check pass.
  • End to end against a test org, emulating workflows/github-actions.yml (--scm github --strict-blocking, same PR_NUMBER logic; the action comes from a real GITHUB_EVENT_PATH payload; real comments posted to a throwaway private PR). Two projects: npm (lodash 4.17.11 + handlebars 4.0.11 baseline, the PR adds vm2 3.9.11) and PyPI (pyyaml 5.3 + django 2.2, the PR adds requests 2.19.1), each with a default-branch baseline. Before = 2.13.0 from PyPI, after = this branch, run twice; both after runs were identical on both projects:
Event / action 2.13.0 (npm / PyPI) This branch, runs 1 and 2 (npm / PyPI)
pull_request / opened, synchronize scanned, exit 1 (35 new + 3 existing / 2 existing) same
pull_request / reopened no scan, exit 0 ("not a supported type") scanned, exit 1, same counts, comment updated
pull_request / ready_for_review no scan, exit 0 scanned, exit 1, same counts
pull_request / closed, edited, labeled no scan, exit 0, INFO no scan, exit 0, WARNING "... is not scanned ... No scan ran"
pull_request, no action in payload no scan, exit 0, INFO "Action None is not a supported type" no scan, exit 0, WARNING "Could not determine the pull_request action"
GITHUB_EVENT_NAME empty no scan, exit 0, ERROR "Unknown event type " no scan, exit 0, WARNING "GITHUB_EVENT_NAME is not set"
pull_request_target / opened no scan, exit 0, ERROR "Unknown event type" no scan, exit 0, WARNING "runs against the base branch"
merge_group, schedule no scan, exit 0, ERROR no scan, exit 0, WARNING listing handled events, issue_comment (ignore commands)
push on the PR head commit (PR_NUMBER=0), workflow_dispatch full scan, exit 0 same (the push/pull_request overlap the new docs section describes)
issue_comment / created comment flow, exit 3 same (pre-existing; this PR doesn't touch the issue_comment path)

Other open 2.13.x fixes

2.13.1 and 2.13.2 were released from main on 2026-11-20, so this branch merges main at v2.13.2 and bumps to 2.13.3 in pyproject.toml, socketsecurity/__init__.py and uv.lock, with its CHANGELOG entries under a new ## 2.13.3 heading. #459, #460, #461, #462, #463, #464 and #465 all make the same bump, so the version files merge cleanly between them, and once one merges the rest ride main's unreleased 2.13.3 (Version Check accepts a version equal to main's while it is above the latest PyPI release). Merging a second one conflicts only in CHANGELOG.md: keep one ## 2.13.3 heading and combine the bullets under its subsections (Upgrade notes, Changed, Fixed, Documentation, in that order). There are no code conflicts between them. If main releases 2.13.3 before this merges, bump to the next patch again.

Public Changelog

With --scm github, reopened pull requests and pull requests marked ready for review are scanned. They used to exit 0 without scanning, which could pass a required check on a commit an earlier run failed. Skipped GitHub events and GitLab pipeline sources now log a warning that no scan ran.

With --scm github, a pull_request run whose action was not opened or
synchronize logged "Pull Request Action reopened is not a supported type"
and exited 0 without scanning. The example workflow subscribes to
reopened, and a required check takes the latest run on the head commit,
so reopening a pull request could replace a failed Socket check with a
pass on the same commit.

reopened and ready_for_review now run the same comparison as opened and
synchronize. Other actions and unhandled events are still skipped with
exit 0, as the exit-code contract documents, but log a warning that no
scan ran. pull_request_target gets its own message because it runs
against the base branch, so it is not treated as pull_request. GitLab's
unhandled pipeline sources log the same warning.
…'t scan

A missing pull_request action or GITHUB_EVENT_NAME (or CI_PIPELINE_SOURCE on
GitLab) now gets its own warning instead of "action 'None' is not scanned",
whose advice to drop the action from the workflow didn't apply. The skip
warnings list issue_comment as handling ignore commands, not as a scanned
event. Tests pin push, workflow_dispatch and issue_comment routing.
Move the Skipped events section below the scan-type paragraphs it had split.
State the required-check risk without claiming which run GitHub picks.
Explain that push on every branch also runs the job on a pull request's head
commit as a full scan that exits 0, and give two ways to keep that run from
satisfying a required check.
2.13.2 was released from main, so this change needs the next patch version.

This branch was successfully deployed

1 active deployment
socket-firewall — 9b086740 Deployed Nov 20, 2026 by dc-larsen via python-sfw-smoke-enterprise #663
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants