TerraQuest: Put the Phone in Your Pocket, Let an Open-Weight Model Name the Animals and Plan
Описание и применение
TerraQuest позволяет гулять без отвлечения на экран, автоматически отслеживает маршрут, озвучивает задания и распознаёт растения и животных с помощью модели CLIP, сохраняя результаты в журнале.
Задача
Outdoor apps keep users staring at screens, interrupting the walking experience and hindering learning about nearby nature.
Результат
TerraQuest enables uninterrupted walking, provides spoken missions, identifies wildlife offline or online, records walks, and offers a post‑walk journal, fostering a habit of mindful nature exploration.
Рабочий процесс
- Start walk – GPS route and timer run in background
- Walk and take a photo of a plant, bird, insect or animal
- If online, send photo to serverless function that calls a cloud vision model
- If offline or cloud fails, run CLIP ViT‑B/32 (Xenova/clip‑vit‑base‑patch32) in the browser via Transformers.js
- Score photo against ~70 nature labels tuned for Indian walks
- Return name and scientific name, create result card
- Save photo, GPS, time, and name to journal
- Pin result on map and update route
- Generate next mission
- Stop walk – generate walk report
Доказательства
The first scan downloads the on‑device model once. After that, scanning works without a connection.
Walks track themselves. GPS distance, active time and estimated calories run without you touching anything.
Missions are read aloud. Prompts like "listen for a bird for 10 seconds" are spoken, and you can answer by voice.
- Дата записи: 2026-10-08T21:56:04.048Z