How we use Jev to answer quiz questions (a 90-question benchmark)

说明与用途

文章说明了如何将Jev集成到QuizPilot Chrome扩展中以回答测验问题。Jev在0.17–0.44秒内实现了90道题中85–86道正确答案，速度更快，准确性更高，并提供可靠的置信度信号。

问题

Integrating a decision model (Jev) into a quiz‑answering Chrome extension to improve accuracy and speed over general LLMs.

结果

Jev answered 85–86 of 90 questions correctly across two runs, outperforming a lightweight LLM (60/90) and matching a reasoning LLM (90/90) while being 15–50× faster.

工作流

• 1. Identify question type (single choice, true/false, multi‑select).
• 2. Convert to Jev format: single choice → categorical, true/false → yes/no, multi‑select → multiple yes/no questions.
• 3. Send a single request containing all questions to Jev via OpenRouter.
• 4. Receive choice probabilities and compute confidence (|2p‑1| for yes/no, lowest confidence for multi‑select).
• 5. If confidence < 0.6, flag for re‑check or fallback to lightweight LLM.
• 6. Aggregate results and present suggested answers in the extension UI.

证据

Jev answered 85–86 of 90 questions correctly across two runs.

来源

one Jev request answered 60 questions in 0.17–0.44 s.

来源

All nine mistakes were single-choice questions whose answer has to be computed, not recognised.

来源

Re-checking everything below 0.6 would have touched 14 of 180 answers (8%).

来源