How we use Jev to answer quiz questions (a 90-question benchmark)

Description and use

The article details how Jev was integrated into the QuizPilot Chrome extension to answer quiz questions. Jev achieved 85–86 of 90 correct answers in 0.17–0.44 s, outperforming general LLMs in speed and providing reliable confidence signals.

Problem

Integrating a decision model (Jev) into a quiz‑answering Chrome extension to improve accuracy and speed over general LLMs.

Outcome

Jev answered 85–86 of 90 questions correctly across two runs, outperforming a lightweight LLM (60/90) and matching a reasoning LLM (90/90) while being 15–50× faster.

Workflow

• 1. Identify question type (single choice, true/false, multi‑select).
• 2. Convert to Jev format: single choice → categorical, true/false → yes/no, multi‑select → multiple yes/no questions.
• 3. Send a single request containing all questions to Jev via OpenRouter.
• 4. Receive choice probabilities and compute confidence (|2p‑1| for yes/no, lowest confidence for multi‑select).
• 5. If confidence < 0.6, flag for re‑check or fallback to lightweight LLM.
• 6. Aggregate results and present suggested answers in the extension UI.

Evidence

Jev answered 85–86 of 90 questions correctly across two runs.

Source

one Jev request answered 60 questions in 0.17–0.44 s.

Source

All nine mistakes were single-choice questions whose answer has to be computed, not recognised.

Source

Re-checking everything below 0.6 would have touched 14 of 180 answers (8%).

Source