I built the assistant I clicked on as a kid. It haunts me now!

Описание и применение

Paperclip – локальный десктопный помощник для macOS, который наблюдает за активным окном, использует модель Gemma 4 через Ollama, сохраняет состояние с помощью Temporal и никогда не отправляет скриншоты внешним серверам.

Задача

A user wanted a desktop assistant that could observe the screen locally, offer help, and persist across restarts without sending data externally.

Результат

Paperclip provides a local, persistent, privacy‑preserving desktop assistant that observes the screen, offers context‑aware help, and survives restarts without external data leakage.

Рабочий процесс

• Clone repo
• Run npm run setup to install Temporal, Ollama, and dependencies
• Configure .env and macOS permissions
• Start ghost with npm run ghost:start
• Ghost checks active window every 5 s
• When window changes, take screenshot, resize, run Gemma 4 via Ollama
• Parse JSON output {app,activity,detail,notable}
• If stuck >20 min, offer help
• Persist state in Temporal workflow
• Restart ghost via launchd if killed

Доказательства

Paperclip runs a local vision model through Ollama, so nothing about your screen leaves your machine.

Источник

Each glance is one invoke_agent span in Sentry, with every step nested under it.

Источник

The biggest speed win came from asking for less. Instead of a paragraph, the model now returns a small JSON object: {app, activity, detail, notable}.

Источник