I Found Two Bugs in a Hackathon's Judging Tool. Neither Explained Why I Lost

Description and use

The author discovered two bugs in the hackathon judging tool, fixed them, recovered a valid accuracy score, and improved the total score, but the low originality score still caused elimination.

Problem

The judging tool’s accuracy calculation silently failed, causing a zero score for the accuracy component and a low originality score, leading to elimination.

Outcome

The author fixed two tooling bugs, recovered a valid accuracy score, and improved the total score significantly, but the originality gate still caused elimination. The case highlights the importance of understanding all scoring components, including hidden originality checks.

Workflow

• 1. Clone all semifinalist repos and review technical reports.
• 2. Inspect submission.json; notice empty accuracy array.
• 3. Identify bugs in adtc_profiler accuracy.py (incorrect base_url) and lm-eval-harness GGUF backend (echo/logprobs mismatch).
• 4. Patch both locally: use proper http://host:port URL and generate exact answer text to read confidence scores.
• 5. Run hackathon benchmark to obtain real accuracy score.
• 6. Update submission.json with calculated accuracy.
• 7. Recalculate total score.
• 8. Submit updated results.
• 9. Receive rejection due to low originality score.
• 10. Reflect on scoring formula and originality gate.

Evidence

accuracy: []

Source

Two bugs, both confirmed by reading the actual source

Source

arc_easy acc_norm = 0.74

Source

Originality Score (0–10): 3

Source