I Found Two Bugs in a Hackathon's Judging Tool. Neither Explained Why I Lost

说明与用途

作者在黑客松评估工具中发现并修复了两个错误，获得了真实的准确率，但原创性门槛仍未通过，导致被拒绝。

问题

The author’s submission was rejected because the scoring formula penalized missing accuracy data and originality, despite the model being a stock RAG architecture that performed competitively.

结果

The author’s debugging effort raised the accuracy score dramatically, but the originality gate remained a blocker. The case illustrates the importance of understanding all scoring components, including hidden gates.

工作流

• Built StacksNG with Qwen2.5‑Coder‑7B and a 780‑chunk RAG corpus.
• Submitted the model with an empty accuracy array.
• Received rejection email citing low originality and missing accuracy.
• Cloned all semifinalist repos and noted that all used fine‑tuned models.
• Identified two bugs in adtc_profiler and lm-evaluation-harness that prevented accuracy calculation.
• Patched the bugs locally and reran the benchmark to obtain a real accuracy score.
• Re‑submitted the updated accuracy data.
• Still failed due to originality gate.
• Analyzed scoring formula and discovered the originality gate was not reflected in the rubric.
• Concluded that a model modification (e.g., LoRA) would be required to pass the originality gate.

证据

My own submission.json:

"accuracy": []

来源

I patched both locally. Instead of trusting the broken echo behavior, I forced the model to generate the exact answer text (a grammar rule constraining what it's allowed to output) and read its confidence scores off that generation directly.

来源

Originality Score (0–10): **3**

来源