Pre-registered result - your scanner misses half of self-hosted apps' own security advisories

Description and use

Dependency scanners frequently miss self-hosted applications’ own advisories, creating blind spots in vulnerability detection.

Dependency scanners often miss security advisories published by self-hosted applications, leading to blind spots in vulnerability detection.

Self-hosted applications have a significantly higher rate of scanner‑blind advisories (53.0%) compared to libraries (0%).

Select 40 top self-hosted applications and 40 top libraries; ensure no overlap with previous samples.
Identify published repository advisories at least 14 days old.
Determine scanner‑blind advisories: those with no OSV record naming a package, searched by GHSA ID and CVE aliases.
Perform a two‑sided permutation test (10,000 permutations, fixed seed) to compare blind rates between groups.
Report results and confidence intervals.
Spot‑check individual advisories (e.g., Vaultwarden GHSA-h6cc-rc6q-23j4).

The split holds.

Applications minus libraries is +53.0 points, p = 0.0057.

If you self-host something, don't expect your dependency scanner to warn you about that application's own advisories.