Using JSON files instead of a database for 1,699 exam paper records

说明与用途

使用 JSON 文件代替数据库管理 1,699 条考试论文记录导致 slug 生成、分析读取和并发写入中的隐藏错误。通过实现冲突感知的 slug 生成、写入队列和显式错误处理，网站现在能够正确提供所有记录并防止文件损坏。

问题

Managing 1,699 exam paper records with flat JSON files caused hidden bugs in slug generation, analytics reading, and concurrent writes, leading to unreachable records, broken homepage sections, and data corruption.

结果

The site now serves all 1,699 exam papers correctly, displays accurate download analytics, and prevents file corruption from concurrent writes. The main limitations are process count and the need for manual index rebuilding.

工作流

• 1. Read entire JSON file with fs.readFileSync and parse it.
• 2. Generate slugs using a base slug and, if necessary, a code‑qualified suffix to avoid collisions.
• 3. Build a slug index keyed by file mtime and size to avoid rebuilding on every call.
• 4. Enqueue write operations via a promise chain to serialize concurrent writes.
• 5. Write to a temporary file and rename to avoid partial writes.
• 6. Restore from a backup if JSON parsing fails.
• 7. Replace silent defaults (|| {}) with explicit error handling for missing keys.

证据

95 papers that existed but had no URL

来源

Two requests writing the file at once will corrupt it

来源