A Digest Quality Score That Knows What It Can't Prove

说明与用途

GistMyMail 添加了一个确定性质量评估器，用于检查 AI 生成的摘要的来源覆盖、事实依据、可操作性、重复内容和信息完整性，并将详细报告附加到每个摘要中。

问题

AI‑generated email digests can present claims that are not supported by the source, leading to false certainty.

结果

The evaluator provides deterministic, model‑agnostic quality metrics for AI‑generated digests, exposing gaps in source coverage and factual grounding without relying on an LLM judge.

工作流

• 1. Generate a digest with GistMyMail.
• 2. The evaluator runs after generation, using the fetched messages and evidence.
• 3. It computes component scores (coverage, factual grounding, actionability, duplicate content, information completeness) and normalizes weights.
• 4. It produces a quality report with rubric version, component scores, counts, statuses, warnings, and source references.
• 5. The report is persisted with the digest and returned by the digest API.
• 6. Reviewers inspect the report via Developer Tools → Network → POST /api/digest/generate → digest.qualityReport.

证据

GistMyMail's v1 quality evaluator makes several checks explicit and repeatable.

来源

The evaluator changes described below are currently in local working-tree changes and are not yet on the public GitHub branch.

来源

The evaluator runs after digest generation using the messages and evidence already fetched.

来源

The report is persisted with the digest and returned by the existing digest API.

来源