Text and Images in One Embedding Space on a CPU: What I Measured Building a Semantic grep

Описание и применение

anygrep позволяет искать заметки, PDF‑файлы и изображения по смыслу, используя EmbeddingGemma 2 в формате int8 ONNX на CPU. Инструмент поддерживает смешанные языки, точный поиск строк и настраиваемый бюджет токенов для изображений, обеспечивая быстрый поиск без GPU.

Задача

Users need to find notes, PDFs, and images by meaning rather than exact words, especially when only the topic is remembered.

Результат

anygrep achieves semantic search across text, PDFs, and images on a CPU, with int8 ONNX inference providing reasonable speed (≈1.5 s per search on a demo corpus). It handles mixed languages and exact string queries, and offers configurable image token budgets for speed/accuracy trade‑offs.

Рабочий процесс

• Install anygrep via pipx
• Run anygrep with a query and a directory to index
• anygrep loads the int8 ONNX model, processes text and images into embeddings
• It compares query embeddings to indexed embeddings using cosine similarity
• Results are ranked and displayed, with exact string matches prioritized when appropriate

Доказательства

anygrep stores paragraphs, PDF pages and pictures in one index, compares each with the query using cosine similarity, and shows them in one ranked list.

int8 was about 2x faster than fp32 for text and about 1.4x faster for images.

anygrep takes about 213 MB. The model files (about 540 MB: 346 MB for text, 195 MB for images) are downloaded on first use.

The current rule needs two things: a minimum score per query/result type pair, and a margin above the query's reference level.