SafePlate: a menu checker for my friend with a peanut allergy, built on Gemma

Описание и применение

SafePlate автоматически классифицирует блюда из меню по аллергенам, предоставляя конкретные вопросы для уточнения и голосовые подсказки, что снижает риск аллергических реакций.

Задача

Akshitha has a severe peanut, tree nut, and shellfish allergy and repeatedly experiences allergic reactions when eating out because allergens are hidden in menu items.

Результат

SafePlate processes a menu photo in about 13 seconds on a laptop, classifying dishes with high accuracy and providing actionable server questions. The safety net ensures no allergen keyword is missed, preventing false safe verdicts.

Рабочий процесс

• 1. Take a photo of the menu with a phone or laptop camera.
• 2. Upload the photo to the SafePlate web page.
• 3. Gemma 3 (4B) processes the image locally via Ollama, returning structured JSON with verdict, reason, and ask_server fields.
• 4. A keyword safety net scans dish text for allergen keywords and forces an 'avoid' verdict if any are found.
• 5. The UI displays each dish with color-coded verdicts and the exact question to ask the server.
• 6. Optionally, tap the microphone icon to hear the verdicts spoken aloud.

Доказательства

SafePlate tells your friend which dishes to avoid, which to ask about, and which look safe for their allergies.

Источник

Gemma reads the menu photo itself and I don't need a separate OCR step.

Источник

A safety net that doesn't trust the model. This is the part I care about most.

Источник