Lazy CI Triage and the limits of a correct citation

说明与用途

Lazy CI Triage CLI 自动提取失败测试数据并进行分类，但模型相较规则的额外价值尚未得到证明。

问题

Automating the extraction and categorization of failed test information while keeping evidence separate from model interpretation.

结果

The tool reliably extracts test metadata and produces validated reports, but model‑added value over rule‑based selection is unproven; both approaches achieve similar validation success.

工作流

• 1. Run the CLI with a test log file and desired format (e.g., python -m lazy_ci_triage fixtures/lexical-trap.log --format pytest --public-data --mode inventory --output reports/article-inventory).
• 2. The tool parses the log, extracts test identifiers and failure details.
• 3. A local model selects a quote and tentative category based on finite choices.
• 4. A validator checks that the quote matches the record and that the category is supported; otherwise it abstains.
• 5. The report is written, separating Python observations, model selection, selected quote, and uncertainty.
• 6. Users review the report to decide on debugging actions.

证据

Lazy CI Triage is a small experimental Python CLI. It reads JUnit XML or supported pytest short summaries and makes an index of failed tests.

来源

We have not measured human time savings or shown that the model adds value over simple rules.

来源

The separate six-record stress test was poor: only 1/5 earlier and 2/5 revised reports passed validation.

来源

The recorded runs used Qwen2.5-Coder 1.5B Instruct Q4_K_M through Ollama 0.18.0, on an RTX 3090 with 24 GiB VRAM and a Ryzen 7 5800X.

来源