I used JSON files instead of a database for 1,699 records. Here’s what actually broke

Description and use

Using flat JSON files instead of a database led to slug collision issues, hidden bugs, and file corruption risks during concurrent writes.

Problem

Relying on flat JSON files for a read‑heavy, append‑mostly dataset caused hidden data integrity and scalability issues.

Outcome

The project remained functional but suffered from 95 unreachable papers, 52 redirect loops, a missing homepage section, and potential file corruption under concurrent writes.

Workflow

• Read entire JSON file on each request
• Filter array for queries
• Write entire file on updates
• Generate slugs from paper metadata
• Build slug index to detect collisions
• Enqueue writes to serialize concurrent writes
• Rename temp file to replace original
• Restore from backup on parse failure

Evidence

95 papers that existed but had no URL

Source

Two requests writing the file at once will corrupt it

Source