Text and Images in One Embedding Space on a CPU: What I Measured Building a Semantic grep

说明与用途

anygrep 通过在 CPU 上使用 EmbeddingGemma 2 的 int8 ONNX 模型，实现对笔记、PDF 和图像的语义搜索。它支持多语言、精确字符串匹配和可配置的图像 token 预算，提供约 1.5 秒的搜索速度。

问题

Users need to find notes, PDFs, and images by meaning rather than exact words, especially when only the topic is remembered.

结果

anygrep achieves semantic search across text, PDFs, and images on a CPU, with int8 ONNX inference providing reasonable speed (≈1.5 s per search on a demo corpus). It handles mixed languages and exact string queries, and offers configurable image token budgets for speed/accuracy trade‑offs.

工作流

• Install anygrep via pipx
• Run anygrep with a query and a directory to index
• anygrep loads the int8 ONNX model, processes text and images into embeddings
• It compares query embeddings to indexed embeddings using cosine similarity
• Results are ranked and displayed, with exact string matches prioritized when appropriate

证据

anygrep stores paragraphs, PDF pages and pictures in one index, compares each with the query using cosine similarity, and shows them in one ranked list.

int8 was about 2x faster than fp32 for text and about 1.4x faster for images.

anygrep takes about 213 MB. The model files (about 540 MB: 346 MB for text, 195 MB for images) are downloaded on first use.

The current rule needs two things: a minimum score per query/result type pair, and a margin above the query's reference level.