I Built an AI Judge That Doesn't Decide Who Wins

Description and use

FairJudge AI provides transparent, evidence-based scoring for hackathon projects, linking scores to evidence, identifying missing evidence, comparing judges, detecting significant disagreements, and recommending human review while keeping final decisions with human judges.

Problem

Hackathon judges often give inconsistent scores because they interpret criteria differently, lack evidence, or emphasize different aspects of a project.

Outcome

FairJudge AI standardizes scoring, increases transparency, and flags disagreements for human review, while keeping final decision with human judges.

Workflow

• Collect project and rubric
• Generate evidence-based AI evaluation
• Validate evidence
• Apply deterministic Python scoring
• Compare multiple judges
• Detect significant disagreements
• Recommend human review

Evidence

FairJudge is an AI-assisted hackathon evaluation system designed to make judging more transparent, evidence-based, and consistent.

Source

It can: Evaluate projects against a supplied rubric; Connect scores to supporting evidence; Identify missing evidence; Explain why a score was given; Compare multiple judges; Detect significant scoring disagreements; Recommend human review when judges disagree.

Source

It does not decide who the objectively correct winner is. The final decision always remains with human judges.

Source