Our Experience at the Hacktober Fest

Description and use

During a hackathon, the team built a tool that generates and tests multiple hypotheses to explain sales drops, rather than relying on a single confident AI answer.

Problem

AI tools often give a single confident answer that may be incorrect, making it hard for analysts to trust the explanation for business metric changes.

Outcome

In a demo, the obvious reason (a big customer stopping orders) explained only 8% of the sales drop, revealing a more complex underlying cause.

Workflow

• Identify a business metric drop (e.g., sales).
• Generate multiple possible explanations using an AI model.
• Retrieve actual sales data for each hypothesis.
• Test each hypothesis against the data to calculate its explanatory power.
• Report the percentage of the drop explained by each hypothesis and highlight the most supported ones.

Evidence

Our team, Training with Vibes, built something called Argue With My Data. The idea came from a simple frustration: you ask an AI tool why your sales dropped, and it gives you one confident answer that sounds right but might be completely off.

Source

It comes up with several possible explanations, tests each one against the actual numbers, and tells you which ones hold up and which don't.

Source

In our demo, the "obvious" reason, a big customer who stopped ordering, turned out to explain only 8% of the drop.

Source

One version of our code passed every test on one laptop and crashed on another.

Source

An AI call froze in the middle of testing and we had to figure out why.

Source