I Found Two Bugs in a Hackathon's Judging Tool. Neither Explained Why I Lost

说明与用途

作者发现并修复了黑客松评审工具中的两个错误，恢复了准确度得分并显著提升总分，但低原创性得分仍导致被淘汰。

问题

The judging tool’s accuracy calculation silently failed, causing a zero score for the accuracy component and a low originality score, leading to elimination.

结果

The author fixed two tooling bugs, recovered a valid accuracy score, and improved the total score significantly, but the originality gate still caused elimination. The case highlights the importance of understanding all scoring components, including hidden originality checks.

工作流

• 1. Clone all semifinalist repos and review technical reports.
• 2. Inspect submission.json; notice empty accuracy array.
• 3. Identify bugs in adtc_profiler accuracy.py (incorrect base_url) and lm-eval-harness GGUF backend (echo/logprobs mismatch).
• 4. Patch both locally: use proper http://host:port URL and generate exact answer text to read confidence scores.
• 5. Run hackathon benchmark to obtain real accuracy score.
• 6. Update submission.json with calculated accuracy.
• 7. Recalculate total score.
• 8. Submit updated results.
• 9. Receive rejection due to low originality score.
• 10. Reflect on scoring formula and originality gate.

证据

accuracy: []

来源

Two bugs, both confirmed by reading the actual source

来源

arc_easy acc_norm = 0.74

来源

Originality Score (0–10): 3

来源