Provenance in the Primary Key: 54 Facts In, 49 Stored, and a Dedupe That Has Never Fired

说明与用途

分析引擎的主键包含收集器和定位器，导致跨收集器去重失败，导致图表和报告中的数据不一致。

问题

The Analysis Engine’s deduplication logic, based on a composite primary key that includes collector and locator, fails to merge duplicate facts from different collectors, leading to inconsistent data in downstream graph construction.

结果

The deduplication logic never fired for cross‑collector duplicates because the primary key included collector and locator, causing each source to be treated as a separate fact. The graph layer’s merge logic inadvertently masked the issue, but the weaker data persisted in reports.

工作流

• 1. Collect observations from repo collector.
• 2. Store observations with composite primary key.
• 3. Deduplicate by comparing confidence on key collisions.
• 4. Build SystemGraph by merging component observations by key.
• 5. Canonicalise provider resources into logical names.
• 6. Generate reports from the graph.

证据

Line 82 of my Analysis Engine's fact base says it deduplicates and keeps the most confident copy.

来源

The five collisions were all config rows for HTTP routes, each the same line of services/api/main.py reported twice by the same scan at the same confidence, 0.85 against 0.85.

来源

For PostHog it returns the 0.60 observation from web/app.tsx:2, not the 0.75 one from web/package.json.

来源

The analyzer builds a SystemGraph from the component observations, and add_component() merges by key: the first copy becomes the node, later copies fill in any attribute the node is missing, and their evidence locators are appended to a list.

来源