Using JSON files instead of a database for 1,699 exam paper records

说明与用途

使用JSON文件代替数据库管理1,699份考试试卷记录导致URL生成、数据一致性和写入并发的隐藏错误。解决方案包括冲突感知slug生成、写入队列和原子文件写入。

问题

Managing a large set of exam papers with flat JSON files caused hidden bugs in URL generation, data consistency, and write concurrency.

结果

The project remained functional with flat files, but required significant code changes to avoid slug collisions, data drift, and write corruption. The solution improved reliability while keeping the architecture simple.

工作流

• 1. Read entire JSON file with fs.readFileSync and parse.
• 2. Generate slugs using a base slug and, if necessary, a code‑qualified suffix.
• 3. Build a primary index keyed by base slug to detect collisions.
• 4. Enqueue write operations to serialize file writes and avoid corruption.
• 5. Use temp file + rename for atomic writes and backup on failure.
• 6. Perform a full sitemap sweep after any URL‑related change to catch broken links.
• 7. Replace silent defaults (|| {}) with explicit error handling for missing keys.

证据

95 papers that existed but had no URL

来源

Two requests writing the file at once will corrupt it

来源