Lazy CI Triage and the limits of a correct citation

Описание и применение

CLI‑инструмент Lazy CI Triage автоматически извлекает данные о неудачных тестах и классифицирует их, но эффективность модели над правилами не доказана.

Задача

Automating the extraction and categorization of failed test information while keeping evidence separate from model interpretation.

Результат

The tool reliably extracts test metadata and produces validated reports, but model‑added value over rule‑based selection is unproven; both approaches achieve similar validation success.

Рабочий процесс

• 1. Run the CLI with a test log file and desired format (e.g., python -m lazy_ci_triage fixtures/lexical-trap.log --format pytest --public-data --mode inventory --output reports/article-inventory).
• 2. The tool parses the log, extracts test identifiers and failure details.
• 3. A local model selects a quote and tentative category based on finite choices.
• 4. A validator checks that the quote matches the record and that the category is supported; otherwise it abstains.
• 5. The report is written, separating Python observations, model selection, selected quote, and uncertainty.
• 6. Users review the report to decide on debugging actions.

Доказательства

Lazy CI Triage is a small experimental Python CLI. It reads JUnit XML or supported pytest short summaries and makes an index of failed tests.

Источник

We have not measured human time savings or shown that the model adds value over simple rules.

Источник

The separate six-record stress test was poor: only 1/5 earlier and 2/5 revised reports passed validation.

Источник

The recorded runs used Qwen2.5-Coder 1.5B Instruct Q4_K_M through Ollama 0.18.0, on an RTX 3090 with 24 GiB VRAM and a Ryzen 7 5800X.

Источник