Grassdex: Building a 100% Offline Nature Dex & Touch-Grass Bingo with Local Gemma

Описание и применение

Grassdex – офлайн‑приложение для распознавания природы, использующее локальную модель Gemma 4 через Ollama. Оно быстро идентифицирует объекты, генерирует 3×3‑карту «Touch‑Grass Bingo» и ограничивает экранное время до 4 минут за 45‑минутную прогулку.

Задача

Outdoor nature identification apps rely on cloud servers, causing long wait times, privacy concerns, and loss of service in low‑signal areas.

Результат

Grassdex achieved 65 % strict accuracy and 94 % lenient accuracy on 17 test snaps, with 0 % JSON parse failures and median inference latency of 16.1 s. The app kept screen time under 4 min during a 45‑min walk.

Рабочий процесс

• 1. Phone camera captures image and sends it over local Wi‑Fi to the laptop.
• 2. Flask app forwards image to Ollama running Gemma 4.
• 3. Single model call returns JSON with species name, confidence, facts, and bingo verification.
• 4. UI displays result and updates bingo card; when a line is completed, it shows "Bingo! Now put the phone away."

Доказательства

Grassdex is a zero‑cloud local web app where your phone camera snaps nature over a private hotspot to your laptop, where an open‑weight vision model (Gemma 4 via Ollama) names the species and checks off a model‑generated 3x3 touch‑grass bingo card.

Источник

Total screen‑on time was under 4 minutes across the entire 45‑minute walk (< 9% of total time).

Источник

Strict accuracy (exact species/subject): 65% (11/17, Wilson 95% CI: 41% – 83%).

Источник

Gemma’s raw confidence scores are uncalibrated… When Gemma 4 was dead wrong, it was just as confident as when it was right.

Источник