Text and Images in One Embedding Space on a CPU: What I Measured Building a Semantic grep

Description and use

anygrep enables semantic search across notes, PDFs, and images on a CPU using EmbeddingGemma 2 in int8 ONNX format. It supports mixed languages, exact string lookup, and configurable image token budgets for speed/accuracy trade‑offs, delivering searches in about 1.5 s on a demo corpus.

Problem

Users need to find notes, PDFs, and images by meaning rather than exact words, especially when only the topic is remembered.

Outcome

anygrep achieves semantic search across text, PDFs, and images on a CPU, with int8 ONNX inference providing reasonable speed (≈1.5 s per search on a demo corpus). It handles mixed languages and exact string queries, and offers configurable image token budgets for speed/accuracy trade‑offs.

Workflow

• Install anygrep via pipx
• Run anygrep with a query and a directory to index
• anygrep loads the int8 ONNX model, processes text and images into embeddings
• It compares query embeddings to indexed embeddings using cosine similarity
• Results are ranked and displayed, with exact string matches prioritized when appropriate

Evidence

anygrep stores paragraphs, PDF pages and pictures in one index, compares each with the query using cosine similarity, and shows them in one ranked list.

int8 was about 2x faster than fp32 for text and about 1.4x faster for images.

anygrep takes about 213 MB. The model files (about 540 MB: 346 MB for text, 195 MB for images) are downloaded on first use.

The current rule needs two things: a minimum score per query/result type pair, and a margin above the query's reference level.