Your Demo Dashboard Is Not a Funnel: Auditing Synthetic Events in ClickHouse

说明与用途

SensorFlow 演示仪表板显示的漏斗看似展示了转化，但其数据为合成数据，无法反映真实用户行为。通过检查 ClickHouse 原始行和用户重叠情况可发现没有真实转化。正确的漏斗测量需要身份规则、事件顺序、时间窗口以及过滤掉合成流量。

问题

A demo analytics stack can display a funnel chart that appears to show conversion, but the data is synthetic and does not represent real user behavior, leading to misleading conversion metrics.

结果

The audit demonstrates that synthetic demo dashboards can mislead stakeholders into thinking conversion rates are real. By inspecting raw data, checking user overlap, and applying proper funnel definitions, teams can avoid false metrics and ensure dashboards reflect actual user behavior.

工作流

• 1. Read the demo or seed‑data generator to understand its synthetic nature.
• 2. Query raw rows and distinct IDs for each event stage using ClickHouse SQL.
• 3. Verify overlap between stages before calculating any conversion percentage.
• 4. Add ordered‑event, time‑window, identity, and deduplication rules to the real‑event test.
• 5. Match Superset dashboard filters to the SQL query and exclude synthetic traffic from production reports.

证据

A newly installed analytics stack can show a convincing sequence of "app open → product view → add to cart → purchase" before it has ingested a single real user event.

来源

The current demo SQL inserts rows from numbers(240). It selects the event name with number % 12, and the demo user with number % 48. Because 48 is divisible by 12, each demo-user-* stays in one event category throughout this generated batch.

来源

Recreating the published generator's 240 rows with ClickHouse Local produced 16 / 4 / 0: sixteen viewers, four buyers, and zero IDs in both sets.

来源

First, choose a stable identity rule. Decide whether distinct_id represents a logged-in account, an anonymous browser ID, or a merged identity.

来源