Sulci 1.0: a context‑aware semantic cache for agents, and the number most caches don’t publish

Описание и применение

Sulci 1.0 – контекстно‑осведомленный семантический кэш для LLM‑агентов, сокращающий повторные запросы до 99,7 % в горячих сессиях, с 93,8 % точности кэш‑ответов и 87 мс латентностью.

Задача

LLM agents generate many near‑duplicate queries, causing repeated expensive model calls.

Результат

Sulci 1.0 demonstrates that a lightweight semantic cache can reduce LLM dispatches by up to 99.7% in warm sessions, with 93.8% of cache hits correct and 98.9% correct declines when no answer is cached. Cache hit latency averages 87 ms, dominated by embedding time.

Рабочий процесс

• Clone the Sulci repo and run the benchmark to verify performance.
• Install the library with pip: pip install "sulci[sqlite]". Create a Cache instance: cache = Cache(backend="sqlite", threshold=0.85). For each query: call cache.get(query); if no response, call the LLM and then cache.set(query, response). Optionally pass session_id to enable context‑aware blending. Integrate with existing stacks via one‑line adapters (LangChain, LlamaIndex, LiteLLM, proxy, MCP, TypeScript).

Доказательства

With a cold, empty cache, 33.4% [30.00–35.00%] of dispatches could be served from cache.

Источник

Warm, within the same session, 99.7% [99.69–99.81%].

Источник

Of cache hits, share that are correct 93.8% [92.13–94.69%].

Источник

Cache hit, p50 87ms

Источник

pip install "sulci[sqlite]"

Источник