100 Configurations Give One Reconciliation Total and 20 Spend Totals Spanning 3.66x, So Agreement on the Total Proves Nothing

Description and use

The experiment with 100 financial processing configurations showed that while the reconciliation total stayed constant, spend totals varied over 3.6×, proving that the total alone is an unreliable quality metric. Separate validation of categorisation and transfer handling is essential.

Problem

Different categorisation and transfer handling configurations produce wildly varying spend totals while the reconciliation total remains unchanged, making it hard to trust the total as a quality metric.

Outcome

The study demonstrates that a single reconciliation total can be misleading; spend totals vary dramatically across configurations. Fixing categorisation and transfer handling reduces spend total and pattern error, underscoring the need for independent validation of each component.

Workflow

• 1. Create synthetic transaction dataset with 530 rows, 3 accounts, 27 merchants, including holds, refunds, self‑transfers, and name changes.
• 2. Define 5 categorisers, 5 transfer policies, and 4 netting rules to generate 100 configurations.
• 3. Run each configuration to compute reconciliation total and spend total.
• 4. Observe that reconciliation total is identical across all configurations while spend totals vary widely.
• 5. Apply fixes: (a) transfer‑only fix, (b) categoriser‑only fix, (c) both fixes.
• 6. Compare spend totals and pattern errors for each fix.
• 7. Conclude that reconciliation total alone is insufficient; categoriser and transfer handling must be validated separately.
• 8. Publish results and source code on GitHub (https://github.com/dev48v/money-map).

Evidence

Run all 100 configurations - 5 categorisers x 5 transfer policies x 4 netting rules - and the reconciliation total is identical in every one (₹16,80,080.65) while "what did I spend" comes back 20 different ways spanning 3.66x

Source

The control makes the point twice. reconciled is one cosmetic line on top of an engine that passes 5 of 6 self-checks: the sweep takes it to 6 of 6 by moving 179 assignments worth ₹11.2 lakh and changing the correctness of exactly zero of them.

Source