I built a tutor that circles your math mistake in red pen

Description and use

Somo is a math tutor that photographs handwritten work, identifies the exact line where the student erred, circles it in red, and explains the mistake in the student’s language, followed by a short lesson and practice on real past‑exam questions.

Problem

Students struggle to understand why their handwritten math solutions are wrong, especially when language barriers exist.

Outcome

Somo accurately identifies the wrong line in all three handwritten test problems (halving error, dropped minus sign, unequal multiplication). Diagnosis time reduced from ~116 s to ~10 s with failover and circuit breaker logic.

Workflow

• Snap a photo of the problem and handwritten work
• Gemini reads handwriting and returns structured JSON with transcription, concept, first wrong line, and explanation
• UI circles the wrong line in red
• Student confirms or corrects the transcription
• Somo streams a short lesson covering the concept, error, analogy, worked example, and exam terms
• Student answers a check question; if correct, they practice a real past‑exam question; if wrong, a simpler re‑explanation is provided

Evidence

Gemini reads the handwriting and returns structured JSON with the transcription, the concept, the first wrong line, and why it's wrong.

Source

Failover is a feature. Gemini's free quota ran out mid-build. I routed every call through one function: try Gemini, retry once on a 503/429, then fall back to Claude, with a circuit breaker that skips Gemini for 60 seconds after a quota error.

Source

In testing, Somo found the exact wrong line on all three of my handwritten test problems: a halving error (2B = 8 → B = 3), a dropped minus sign when rearranging, and multiplying the two sides of an equation by different numbers.

Source