Your Demo Dashboard Is Not a Funnel: Auditing Synthetic Events in ClickHouse

Описание и применение

Демонстрационный дашборд SensorFlow показывает «конверсию», но использует синтетические данные, поэтому показатели не отражают реальное поведение пользователей. Проверка исходных строк ClickHouse и пересечение пользователей по этапам демонстрируют отсутствие реальных конверсий. Для корректного измерения необходимо определить правила идентификации, порядок событий, временные окна и исключить синтетический трафик.

Задача

A demo analytics stack can display a funnel chart that appears to show conversion, but the data is synthetic and does not represent real user behavior, leading to misleading conversion metrics.

Результат

The audit demonstrates that synthetic demo dashboards can mislead stakeholders into thinking conversion rates are real. By inspecting raw data, checking user overlap, and applying proper funnel definitions, teams can avoid false metrics and ensure dashboards reflect actual user behavior.

Рабочий процесс

• 1. Read the demo or seed‑data generator to understand its synthetic nature.
• 2. Query raw rows and distinct IDs for each event stage using ClickHouse SQL.
• 3. Verify overlap between stages before calculating any conversion percentage.
• 4. Add ordered‑event, time‑window, identity, and deduplication rules to the real‑event test.
• 5. Match Superset dashboard filters to the SQL query and exclude synthetic traffic from production reports.

Доказательства

A newly installed analytics stack can show a convincing sequence of "app open → product view → add to cart → purchase" before it has ingested a single real user event.

Источник

The current demo SQL inserts rows from numbers(240). It selects the event name with number % 12, and the demo user with number % 48. Because 48 is divisible by 12, each demo-user-* stays in one event category throughout this generated batch.

Источник

Recreating the published generator's 240 rows with ClickHouse Local produced 16 / 4 / 0: sixteen viewers, four buyers, and zero IDs in both sets.

Источник

First, choose a stable identity rule. Decide whether distinct_id represents a logged-in account, an anonymous browser ID, or a merged identity.

Источник