TerraQuest: Put the Phone in Your Pocket, Let an Open-Weight Model Name the Animals and Plan

Description and use

TerraQuest lets users walk without looking at their phone, tracks routes in the background, reads missions aloud, identifies plants and animals with an on‑device CLIP model, and stores findings in a post‑walk journal.

Problem

Outdoor apps keep users staring at screens, interrupting the walking experience and hindering learning about nearby nature.

Outcome

TerraQuest enables uninterrupted walking, provides spoken missions, identifies wildlife offline or online, records walks, and offers a post‑walk journal, fostering a habit of mindful nature exploration.

Workflow

• Start walk – GPS route and timer run in background
• Walk and take a photo of a plant, bird, insect or animal
• If online, send photo to serverless function that calls a cloud vision model
• If offline or cloud fails, run CLIP ViT‑B/32 (Xenova/clip‑vit‑base‑patch32) in the browser via Transformers.js
• Score photo against ~70 nature labels tuned for Indian walks
• Return name and scientific name, create result card
• Save photo, GPS, time, and name to journal
• Pin result on map and update route
• Generate next mission
• Stop walk – generate walk report

Evidence

The first scan downloads the on‑device model once. After that, scanning works without a connection.

Source

Walks track themselves. GPS distance, active time and estimated calories run without you touching anything.

Source

Missions are read aloud. Prompts like "listen for a bird for 10 seconds" are spoken, and you can answer by voice.

Source