A 1.5B model is a triage signal, not a verdict

Description and use

A 1.5 B LLM is used as a triage signal, not a verdict: checks run locally, each output is limited to one question and six lines of context, reducing latency to ~12 s and increasing transparency.

Problem

Auditing trading‑strategy repositories with a 1.5B LLM is too slow for per‑file checks and produces unstable verdicts.

Outcome

The redesigned workflow reduces per‑finding latency to ~12 s, eliminates unstable multi‑sentence outputs, and clearly separates deterministic findings from model‑based confirmation, making the audit practical for nightly runs.

Workflow

• Run deterministic checker to generate findings
• For each finding, extract evidence line + 5 surrounding lines
• Prompt local Qwen2.5‑Coder‑1.5B‑Instruct with yes/no question
• Collect model answers (YES, NO, UNSURE)
• Print report with evidence line and model verdict
• Optionally run --no‑llm for instant deterministic output

Evidence

The tool runs five deterministic checks over the source, then hands each finding to a local model with exactly six lines of context and one yes/no question: is this actually true here?

Source

Four model checks on the bundled sample take about 48 to 50 seconds on this laptop. That is roughly twelve seconds per question.

Source

The model never sees the repository. It sees the finding's evidence line plus five lines around it, and it can answer YES, NO, or UNSURE.

Source

The checker owns the finding. Every path:line in the report comes from code that matched a pattern.

Source