auditsift: security-audit every PR with an AI agent, without drowning in false positives

说明与用途

auditsift 通过先执行离线扫描，然后仅将候选片段发送给两个 LLM 进行验证，并要求引用代码证据，从而降低误报并控制成本，生成 SARIF 报告供 CI 使用。

问题

AI‑based security reviews of PRs often return noisy, untrustworthy findings and incur high costs when run on every pull request.

结果

The workflow reduces false positives by requiring quoted evidence, limits LLM usage to a few snippets, and keeps most checks offline, enabling free audits with optional paid explanations.

工作流

• Run offline "map" phase to detect languages, entry points, routes, dependencies and redact secrets.
• Run offline "scan" phase applying hardcoded rules for secrets, SQL interpolation, shell usage, eval, pickle, yaml.load, weak hashes, verify=False, path traversal, SSRF, permissive CORS.
• Send only candidate snippets to a triage LLM for impact and fix suggestions.
• Send the same snippets to a second, independent verify LLM that must quote existing code before confirming a finding.
• Generate JSON, SARIF 2.1.0, and Markdown reports; use "--fail-on high" to block CI if high‑severity findings remain.
• Filter findings to the PR diff with "--diff" and run the audit in a GitHub Actions job that uploads SARIF to the Security tab.
• Optionally enable LLM passes by setting API keys and model names; use AtmoRouter for cost‑effective access.
• Integrate with agents via "auditsift serve" MCP server to call map/scan/audit only when "use_llm:true".

证据

Ask a coding agent to "review this PR for security issues" and you usually get back a long, confident list. Some of it is real. A lot of it is plausible-sounding noise

来源

auditsift runs an audit in phases. The first two need no API key and never touch the network.

来源

The verify step is the part I care about most. It doesn't see the triage write-up, so it can't just agree with it. A "confirmed" verdict only counts when the quoted evidence is found in the code.

来源

Exit code 1 means a kept finding is at or above the gate, so CI blocks the merge.

来源