SafePlate: a menu checker for my friend with a peanut allergy, built on Gemma

Description and use

SafePlate automatically classifies menu items by allergens, provides specific server questions, and offers spoken guidance, reducing allergy risk.

Problem

Akshitha has a severe peanut, tree nut, and shellfish allergy and repeatedly experiences allergic reactions when eating out because allergens are hidden in menu items.

Outcome

SafePlate processes a menu photo in about 13 seconds on a laptop, classifying dishes with high accuracy and providing actionable server questions. The safety net ensures no allergen keyword is missed, preventing false safe verdicts.

Workflow

• 1. Take a photo of the menu with a phone or laptop camera.
• 2. Upload the photo to the SafePlate web page.
• 3. Gemma 3 (4B) processes the image locally via Ollama, returning structured JSON with verdict, reason, and ask_server fields.
• 4. A keyword safety net scans dish text for allergen keywords and forces an 'avoid' verdict if any are found.
• 5. The UI displays each dish with color-coded verdicts and the exact question to ask the server.
• 6. Optionally, tap the microphone icon to hear the verdicts spoken aloud.

Evidence

SafePlate tells your friend which dishes to avoid, which to ask about, and which look safe for their allergies.

Source

Gemma reads the menu photo itself and I don't need a separate OCR step.

Source

A safety net that doesn't trust the model. This is the part I care about most.

Source