Zero-Token Decisions in the Browser: On-Device Multimodal Classification with EmbeddingGemma 2 and WebGPU

说明与用途

Decide Live 使用 EmbeddingGemma 2 与 WebGPU/WASM 在设备上实现即时、私密、零令牌决策。

问题

Need fast, deterministic, private multimodal decision making without server calls or token generation.

结果

Decide Live achieves deterministic, zero‑token multimodal decisions in the browser with <1 GB model size, no server calls, and privacy by design.

工作流

• 1. Load model weights once and cache them in the browser.
• 2. In a Web Worker, use transformers.js to embed media and query text in a single pass.
• 3. Embed each option once and cache the embeddings.
• 4. Perform mean pooling, L2‑normalize to 768‑dim unit vectors.
• 5. Compute cosine similarity between query and each option, scale by 19.96, and apply softmax to obtain probabilities.
• 6. Display probability bars, current decision, and confidence gate in the UI.
• 7. For audio, process 2‑second windows at 16 kHz mono, skipping windows quieter than –55 dBFS.
• 8. For live camera, score up to four decision cards per frame.
• 9. Run sample image tests to report accuracy and compare prompt formats.

证据

Decide Live produces it with an embedding model and a calibrated similarity, so the number of generated tokens is zero.

来源

Everything below is in the page and the model card, so you can reproduce it.

来源

Weights download once and are cached by the browser. After that, decisions are local.

来源

No decoding loop appears anywhere in that path. One forward pass of the encoder per card, then a dot product.

来源