The model never computes a number

Description and use

<p>My finance app's AI told me things I knew were wrong. Spending far higher than it really was. A charge counted in this month's numbers that hadn't happened in months.</p> <p>Money Track is my personal finance app. It syncs bank statements, sorts spending into categories and has an AI advisor you can ask things like "can I afford this?" On sample data it looked fine. On my real data it started making numbers up.</p> <p>In a finance app, a confidently wrong number is worse than no answer. You act on it.</p> <h2> Telling the model didn't work </h2> <p>My first reaction was the usual one: tell

Problem

<p>My finance app's AI told me things I knew were wrong. Spending far higher than it really was. A charge counted in this month's numbers that hadn't happened in months.</p> <p>Money Track is my personal finance app. It syncs bank statements, sorts spending into categories and has an AI advisor you can ask things like "can I afford this?" On sample data it looked fine. On my real data it started making numbers up.</p> <p>In a finance app, a confidently wrong number is worse than no answer. You act on it.</p> <h2> Telling the model didn't work </h2> <p>My first reaction was the usual one: tell

Outcome

<p>My finance app's AI told me things I knew were wrong. Spending far higher than it really was. A charge counted in this month's numbers that hadn't happened in months.</p> <p>Money Track is my personal finance app. It syncs bank statements, sorts spending into categories and has an AI advisor you can ask things like "can I afford this?" On sample data it looked fine. On my real data it started making numbers up.</p> <p>In a finance app, a confidently wrong number is worse than no answer. You act on it.</p> <h2> Telling the model didn't work </h2> <p>My first reaction was the usual one: tell

Evidence

The model never computes a number

Source