My AI‑vs‑human commit comparison had a control group of one day

说明与用途

AI 生成的提交最初似乎更频繁地包含 "fix"/"revert"，但这主要是由于命名习惯和提交大小。调整后差异消失。

问题

Assess whether AI‑generated commits differ from non‑AI commits in terms of fix/revert subject frequency, but the comparison is confounded by a single‑day control group and commit size.

结果

AI commits initially appear to have a higher fix/revert subject rate, but this is largely due to naming conventions and larger commit size. After adjusting for these factors, the difference disappears.

工作流

• 1. Identify AI commits by Co‑Authored‑By: Claude trailer. 2. Count commits whose subject starts with "fix" or "revert". 3. Compare percentages between AI and non‑AI commits. 4. Expand subject match to include any occurrence of fix/bug/revert. 5. Examine commit size (files touched) and overlap within 72 h. 6. Re‑evaluate metrics after normalizing for commit size and subject wording.

证据

AI commit = the message has a Co‑Authored‑By: Claude trailer (245 commits). Everything else is "no trailer" (65 commits).

来源

subject starts with fix/revert: AI commits 38% vs. no‑trailer 17%

来源

57 of the 65 landed on a single day, June 5.

来源

AI commits touch more files: 3.1 on average, against 1.7.

来源