Skip to content

Keep each advisory when merging duplicate alerts - #134

Draft
David Larsen (dc-larsen) wants to merge 3 commits into
mainfrom
fix/dedupe-keep-each-advisory
Draft

David Larsen (dc-larsen) wants to merge 3 commits into
mainfrom
fix/dedupe-keep-each-advisory

Conversation

@dc-larsen

@dc-larsen David Larsen (dc-larsen) commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Dedupe merges vulnerability alerts that share a type and severity on the same package, even when they are different advisories. fullscans.stream() and purl.post() both run it, so handlebars@4.0.11 comes back with 1 criticalCVE alert instead of 6, and vm2@3.9.11 with 1 instead of 35.

The Python CLI reads existing alerts for --strict-blocking through fullscans.stream() (added in socket-python-cli #453). Two effects from a customer report:

  • The "Existing alerts" PR comment table and the EXISTING blocking issues count list one CVE per package and severity. A scan with 8 Block-action CVEs on unchanged packages reported 3.
  • The merged alert keeps the first alert's action. If that advisory is resolved or triaged to monitor/ignore, the Block action on the other advisories is lost and the CLI exits 0. Reproduced below: 34 Block CVEs present, stock CLI passes.

Root Cause

Dedupe.consolidate_and_merge_alerts keys alerts on (type, severity, category, file, start, end). CVE alerts have no file or position, so every advisory of one severity on a package gets the same key and only the first is kept, along with its action.

Fix

  • Dedupe.alert_key adds the advisory ID (props.ghsaId, then cveId, upper-cased; same lookup order as the CLI's Issue.advisory_id()), and the merge uses it. Different advisories stay separate. Socket's CVE alerts carry only ghsaId and cveId; the other names in ADVISORY_ID_PROPS (ghsa_id, cve_id, cve) mirror the CLI's list, and a code comment says so.
  • The same advisory seen more than once still merges into one alert. Across release files of a package (PyPI wheels and sdist) that happens in purl.post() only: fullscans.stream() rows have no purl/inputPurl, so each release file stays its own artifact there.
  • When alerts do merge, the merged alert keeps the most severe action seen (error > warn > monitor > ignore) instead of the first one. Only the action moves: the merged alert keeps the first sighting's key and props, so in a purl.post() result the key can belong to a release whose own action was less severe than the one shown.
  • An unknown or missing action ranks below ignore. It yields to any known action (ignore included) and never replaces one, so an unexpected value can't downgrade a known error, in either order. Today the API only returns error, warn, monitor and ignore, so this has no current effect.
  • Alerts without an advisory ID merge exactly as before.
  • Version bump to 3.6.4 and changelog entry. 3.6.3 was released from main (Document the API token scopes each SDK method needs #130, Fix settings.integration_events() to request the events path #131) while this PR was open, so main is merged in and this ships as 3.6.4; the 3.6.3 changelog entry is unchanged.

socket-python-cli needs a pin bump to pick this up: SocketDev/socket-python-cli#459, which pins socketdev==3.6.4 and depends on 3.6.4 being published.

Tests

Unit: 10 new tests in tests/unit/test_dedupe.py and tests/unit/test_full_scan_cached_polling_http.py: same-severity advisories stay separate; a triaged-first advisory doesn't hide a later Block one; same advisory across releases merges with the strictest action in any order; action ranking; CVE-only and case handling; non-advisory alerts still merge across releases; unknown actions never replace a known one; the merged alert keeps the first key; a CVE and a non-advisory alert on one package stay separate; empty input. Against main's dedupe.py the advisory and action tests fail (16 failures counting subtests); the empty-input and mixed-alert tests are regression guards and pass on both. All pass here.

  • python -m pytest tests/unit (what CI runs), after merging main: 204 passed, 1 skipped (main before 3.6.3: 193 passed).
  • Full python -m pytest: same 14 failures and 19 errors in tests/integration as on main (they need live credentials); no new failures.
  • hatch build and twine check dist/* pass.

End to end: stock socketsecurity 2.13.0 + socketdev 3.6.2 vs the same CLI with this SDK, against a test org. Each project has a default-branch baseline and a PR branch with no dependency change, run with --strict-blocking --ignore-commit-files --integration api --scm api. "Raw" is counted from GET /orgs/{org}/full-scans/{id} for the same scan: distinct (package, alert type, advisory) with action error. Patched runs were repeated twice with identical results.

Project Build Exit Alerts loaded Existing blockers (CLI) Blockers (raw API) Existing rows in PR comment
npm: lodash 4.17.11 + handlebars 4.0.11 stock 1 49 3 8 3
patched x2 1 68 8 8 8 (6 handlebars rows, each with its own GHSA)
npm: minimist, node-forge, jsonwebtoken, axios, ejs, underscore (old versions) stock 1 61 4 4 4
patched x2 1 117 4 4 4
PyPI: Django 2.2, Jinja2 2.10, requests 2.19.0, PyYAML 5.3, urllib3 1.24.1, Pillow 6.2.0 stock 1 56 3 20 5
patched x2 1 189 20 20 22 (20 Block + 2 Warn)
Public MERN app, 1,545 packages stock 1 2,744 17 22 17
patched x2 1 2,879 22 22 22
npm: vm2 3.9.11, first-listed Critical CVE resolved (ignore) stock x2 0 19 0 34 none
patched x2 1 73 34 34 34
  • In the second npm project every package has at most one Critical CVE, so the blocker count doesn't change. The extra alerts loaded are same-severity High and Medium CVEs (Monitor in the test org) that were collapsed.
  • The few alerts still merged (raw 69 vs 68 loaded, 2,882 vs 2,879) are urlStrings alerts with different URL props and no file, all with action ignore. Their key is unchanged by this PR.
  • purl.post(..., alerts=True) against the live API: handlebars 1 → 6 Critical, Django 2.2 1 → 6, Pillow 6.2.0 1 → 12 (26 release files still merge into one row with no duplicate alerts), vm2 1 → 35.
  • No regression on the new-alerts path: adding vm2 on a PR branch gives the same 35 new-alert rows with both builds, and a non-strict run on an unchanged branch exits 0 with both.

Not changed in this PR

  • The merge rebuilds each alert from a fixed field list (key, type, severity, category, file, start, end, props, action, plus releases). actionSource, alertPolicyRule, fix and reachability are dropped from every alert that goes through fullscans.stream() or purl.post(). That predates this PR and is unchanged by it. Keeping them would mean deciding which sighting's actionSource/alertPolicyRule goes with the winning action and would change the purl.post() output shape, so it's left for a follow-up. The CLI's typed stream(use_types=True) path has no fields for them either.

Public Changelog

fullscans.stream() and purl.post() keep every advisory on a package instead of one alert per type and severity, and a merged alert keeps its most severe action.

Dedupe keyed alerts on type, severity, category, file and position. Every
advisory of one severity on a package shares an alert type (criticalCVE,
cve and so on) and has no file, so they all landed on one key and only the
first survived. fullscans.stream() and purl.post() both go through this, so
handlebars@4.0.11 came back with one criticalCVE alert instead of six.

The merge also kept the first alert's action. When the first advisory of a
group was resolved or triaged to monitor or ignore, the Block action on the
others was lost with them, and a caller that blocks on action == "error"
passed.

The alert key now includes the advisory ID (props.ghsaId, then cveId), and
a merged alert keeps the most severe action seen (error > warn > monitor >
ignore). Alerts without an advisory ID merge as before.

Bumps the version to 3.6.3.
Scope the changelog's cross-release merging to purl.post(): fullscans.stream()
rows carry no purl, so each release file stays its own artifact there.

Note in the changelog and code that a merged alert keeps the first sighting's
key while its action can come from another release, and that the extra
advisory prop names mirror socket-python-cli's ADVISORY_ID_PROPS.

An unknown or missing action keeps ranking below ignore, so it never replaces
a known action and can't downgrade a known error. Tests pin that in both
orders, along with the first-key behavior, a CVE next to a non-advisory alert
on one package, and an empty input. Fix blank-line spacing in the new tests.
3.6.3 was released from main with the settings.integration_events() fix.
Keep that changelog entry as is and ship the advisory dedupe fix as 3.6.4.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants