Andyyyy64/whichllm — Find the local LLM that actually runs and performs best on your hardware.

Description and use

**Brief Summary (2‑3 sentences)**
*whichllm* is a Python‑based utility that scans the models installed on a machine, runs quick, recency‑aware benchmarks, and instantly tells you which local LLM delivers the best performance for your hardware. With a single command it surfaces the top‑ranked model, its API/SDK details, language support and specialized capabilities, letting you add AI functionality without having to start from a blank model stack.

**Value**

• **Speed‑to‑experiment:** Developers can prototype AI features, RAG pipelines, or autonomous agents in minutes instead of days spent hunting for a compatible model.
• **Performance‑driven selection:** Rankings are based on real‑world latency and accuracy benchmarks that are updated regularly, so the chosen model is truly the fastest and most accurate for the current hardware.
• **Low integration friction:** The tool outputs ready‑to‑use API/CLI/SDK signatures and metadata (language, domain focus), making it easy to plug the selected model into existing codebases or orchestration frameworks.

**Practical Adoption Path**

1. **Install & run:** pip install whichllm && whichllm run – the command automatically detects installed models, benchmarks them, and prints the best candidate.
2. **Consume the output:** The CLI returns the model identifier, endpoint details, and a small JSON manifest that can be fed directly into your RAG, agent, or inference pipeline.
3. **Integrate:** Use the provided SDK snippet (Python, REST, or CLI) to replace a placeholder model call in your application, then iterate on prompts or fine‑tuning as needed.
4. **Automate:** Incorporate the whichllm library into CI/CD or provisioning scripts to re‑evaluate model performance after hardware upgrades or new model releases.

**Production Readiness**

• **Community health:** 5 156 stars, 276 forks, recent commits (last update 2026‑06‑23), and active issue discussions indicate a vibrant maintainer community.
• **Stability:** The core functionality is a single‑command benchmark runner with deterministic outputs; no heavy runtime dependencies beyond standard Python and common ML libraries.
• **Ecosystem fit:** Exposes clear API/CLI hooks, language metadata, and topic tags, which align well with existing AI orchestration tools (LangChain, LlamaIndex, etc.).
• **Risks to address:** Formal review of the license (MIT‑like but verify), a security audit of the benchmark execution sandbox, and confirmation of long‑term maintainer commitment before full production rollout.

Overall, *whichllm* offers a high‑confidence, low‑effort path to select the optimal local LLM, making it suitable for pilot projects and, after the minor risk checks, for production‑grade deployments.

Problem

Andyyyy64/whichllm helps add AI capability without starting from a blank model stack.

Outcome

High for an OSS candidate: recent activity, adoption, and ecosystem signals are strong enough for a serious pilot.

Workflow

• prototype AI features
• build RAG or agent workflows
• evaluate model tooling

Evidence

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

Source