I Found Two Bugs in a Hackathon's Judging Tool. Neither Explained Why I Lost

Description and use

The author found two bugs in the hackathon judging tool, fixed them to obtain a real accuracy score, but the originality gate still failed, leading to rejection.

Problem

The author’s submission was rejected because the scoring formula penalized missing accuracy data and originality, despite the model being a stock RAG architecture that performed competitively.

Outcome

The author’s debugging effort raised the accuracy score dramatically, but the originality gate remained a blocker. The case illustrates the importance of understanding all scoring components, including hidden gates.

Workflow

• Built StacksNG with Qwen2.5‑Coder‑7B and a 780‑chunk RAG corpus.
• Submitted the model with an empty accuracy array.
• Received rejection email citing low originality and missing accuracy.
• Cloned all semifinalist repos and noted that all used fine‑tuned models.
• Identified two bugs in adtc_profiler and lm-evaluation-harness that prevented accuracy calculation.
• Patched the bugs locally and reran the benchmark to obtain a real accuracy score.
• Re‑submitted the updated accuracy data.
• Still failed due to originality gate.
• Analyzed scoring formula and discovered the originality gate was not reflected in the rubric.
• Concluded that a model modification (e.g., LoRA) would be required to pass the originality gate.

Evidence

My own submission.json:

"accuracy": []

Source

I patched both locally. Instead of trusting the broken echo behavior, I forced the model to generate the exact answer text (a grammar rule constraining what it's allowed to output) and read its confidence scores off that generation directly.

Source

Originality Score (0–10): **3**

Source