Sulci 1.0: a context‑aware semantic cache for agents, and the number most caches don’t publish

Description and use

Sulci 1.0 is a context‑aware semantic cache for LLM agents that reduces duplicate model calls by up to 99.7% in warm sessions, achieving 93.8% correct cache hits and 87 ms latency.

Problem

LLM agents generate many near‑duplicate queries, causing repeated expensive model calls.

Outcome

Sulci 1.0 demonstrates that a lightweight semantic cache can reduce LLM dispatches by up to 99.7% in warm sessions, with 93.8% of cache hits correct and 98.9% correct declines when no answer is cached. Cache hit latency averages 87 ms, dominated by embedding time.

Workflow

• Clone the Sulci repo and run the benchmark to verify performance.
• Install the library with pip: pip install "sulci[sqlite]". Create a Cache instance: cache = Cache(backend="sqlite", threshold=0.85). For each query: call cache.get(query); if no response, call the LLM and then cache.set(query, response). Optionally pass session_id to enable context‑aware blending. Integrate with existing stacks via one‑line adapters (LangChain, LlamaIndex, LiteLLM, proxy, MCP, TypeScript).

Evidence

With a cold, empty cache, 33.4% [30.00–35.00%] of dispatches could be served from cache.

Source

Warm, within the same session, 99.7% [99.69–99.81%].

Source

Of cache hits, share that are correct 93.8% [92.13–94.69%].

Source

Cache hit, p50 87ms

Source

pip install "sulci[sqlite]"

Source