The Agent Said It Worked. I Asked the Kernel.

说明与用途

实验展示了如何通过独立观察（系统调用、eBPF、CPU 采样、数据包）发现 AI 生成代码中的虚假成功、冗余工作和意外连接。

问题

How to independently verify that an AI‑generated program truly performs its intended task, rather than just reporting success.

结果

The experiment demonstrated that independent, low‑level evidence can expose false success reports, redundant work, and unintended network activity in AI‑generated code, providing a practical verification workflow.

工作流

• 1. Build a minimal native C backup client with checksum logic.
• 2. Create a Python receiver listening on loopback TCP.
• 3. Instrument the client with strace, bpftrace, perf, and tcpdump to capture system calls, kernel events, CPU samples, and packets.
• 4. Run the client twice against the same receiver state to test baseline and changed conditions.
• 5. Verify the saved file’s digest matches the source using an external verifier.
• 6. Analyze collected data to confirm or refute the agent’s claim.
• 7. Identify discrepancies such as missing backups, excessive hashing, or unexpected connections.
• 8. Iterate on the client or agent code based on findings.

证据

The independent verifier finds no saved backup. In the eBPF run, there are no observed client connection calls, and the filtered packet capture contains zero packets.

On the second invocation, the system‑call trace records 134,217,728 logical source bytes read for a 524,288‑byte file: 134,217,728 / 524,288 = 256

The eBPF run observes 256 entries into hash_file on each invocation.