I Watermarked 100 Texts With SynthID and Asked Local LLMs to Wash It Out. The Detector Still Saw Some of It.

说明与用途

本地LLM可部分去除SynthID水印，但均值检测器仍标记23–41%的重写文本。较长文本和小说类更难去除。

问题

Can a local LLM rewrite a SynthID‑watermarked text enough to evade detection while preserving meaning?

结果

Rewriting watermarked texts with local LLMs substantially reduces Bayesian detector flags to near clean‑text levels, but a mean‑test detector still flags 23–41% of rewrites. Longer texts and fiction genres retain more watermark evidence. A second rewrite pass does not further reduce watermark traces.

工作流

• Generate 100 watermarked texts with Qwen3‑4B‑Instruct‑2507 using SynthID config
• Train a Bayesian detector on 1,200 watermarked and 3,480 clean examples
• Generate 100 unwatermarked control texts
• Rewrite each text with qwen3:4b‑instruct, gemma3:12b, and qwen3:14b using a prompt that preserves meaning
• Optionally run a second rewrite pass with gemma3:12b
• Tokenize all texts with Qwen tokenizer
• Run both Bayesian and mean‑test detectors on originals, rewrites, and controls
• Collect flagged percentages, mean g‑values, z‑scores, AUC, and 5‑gram overlap
• Analyze results by model, text length, and genre

证据

The standard detector flagged 3% to 15% of the rewrites, depending on the rewriting model.

来源

The mean test flagged 23% to 41%.

来源

The watermark also shows up fast: using only the first 100 tokens of each original, 94% were flagged; at 200 tokens, 100%.

来源

Longer texts are easier to catch: for 750–900 word texts, qwen3:14b rewrites were flagged 89% of the time.

来源

Fiction is the hardest to wash: short fiction rewrites were flagged 70% by mean test and 47% by Bayesian.

来源