Four reviewers told me the one thing I couldn't fix by myself
说明与用途
作者构建了 DoSync 并提交至 IEEE WF‑IoT 2026。评审指出缺乏独立评估,导致论文被拒。此经历凸显了外部注释者的必要性,并促成了多域基准语料库和更严格的验证。
问题
A solo open‑source project lacked independent evaluation, making it hard to prove claims and satisfy peer review.
结果
The paper was rejected due to insufficient independent evaluation, but the process revealed the need for external annotators and led to a more robust benchmark corpus and third‑party testing.
工作流
- 1. Build DoSync and run self‑judged evaluation on a single registry.
- 2. Submit paper to IEEE WF‑IoT 2026.
- 3. Receive reviews highlighting lack of independent annotation.
- 4. Publish multi‑domain benchmark corpus on GitHub.
- 5. Define precision/recall conventions and perform sensitivity analysis.
- 6. Conduct third‑party testing by installing DoSync on unfamiliar hardware following published instructions.
- 7. Identify defects and refine implementation.
- 8. Seek independent annotators to validate resolver decisions.
证据
The evaluation is small and self‑judged: 15 scenarios with ground truth defined by the author alone.
I couldn't produce one. Every judgement about what the resolver should pick — in all fifteen scenarios, in the paper and in the benchmark files — was mine.
- 记录日期: 2026-09-05T17:30:38.295Z