I built a small Python library to add retries, caching, fallbacks, budgets, and guardrails around native LLM SDK calls

Description and use

callm reduces boilerplate, prevents cost overruns, improves reliability with retries and fallbacks, and enforces privacy and output quality, achieving up to 91% cost savings on FAQ traffic and 100% success rate under 20% random 503s.

Problem

Developers repeatedly wrote boilerplate for LLM calls—retry logic, cost tracking, caching, PII scrubbing, output parsing—leading to duplicated code and hidden costs.

Outcome

callm reduces boilerplate, prevents cost overruns, improves reliability with retries and fallbacks, and enforces privacy and output quality, achieving up to 91% cost savings on FAQ traffic and 100% success rate under 20% random 503s.

Workflow

• Define a function that calls the LLM SDK.
• Decorate the function with @callm, specifying options like cache, retry, fallback, max_cost, block_pii, detect_injection, and output_schema.
• Call the function normally; callm intercepts the call and applies middleware: prompt injection scoring, PII masking, cache lookup, validation, fallback, cost guard, and retry.
• The function returns a validated Pydantic model or an OpenAI ChatCompletion object, unchanged from the SDK.
• Use callm stats to monitor tokens, cost, cache hits, and retries.
• Optionally run the offline demo or install via pip.