Ten full catalogues in a day: what the arr ecosystem's own backends will and will not let a directory carry

Description and use

Built a catalog of 90,000+ TV shows, 350,000+ games, and more, sourced from open APIs that allow commercial use with attribution.

Problem

Determine which public metadata backends allow a commercial directory to store full data with attribution.

Outcome

Successfully built a commercial catalog of 90,000+ TV shows, 350,000+ games, and additional media, all sourced from licenses permitting commercial use with attribution.

Workflow

• 1. Reviewed each backend’s terms to assess commercial use and attribution requirements.
• 2. Identified compliant sources (TVmaze, IGDB, Wikidata, TheSportsDB, Kitsu, Project Gutenberg, LibriVox, Steam).
• 3. Built one agent per source to fetch data, store responses as test fixtures, and probe live APIs.
• 4. Created a second agent per source to validate the first: rerun tests, compare consecutive pulls, verify license compliance.
• 5. Corrected seven of eight adapters based on manual review of license text and API behavior.
• 6. Implemented capped, resumable pagination with cursors and scheduled re‑runs.
• 7. Published catalogues at nichedb.dev and documented the pattern at logicsrc.com/asdlc.

Evidence

Every show TVmaze knows, close to 90,000, CC BY-SA, walked at 250 a page and then kept current from its updates feed.

Source

Every game IGDB knows, about 350,000; IGDB says in its own words that it prefers the data stored and served.

Source

Every film and every video game on Wikidata, CC0, walked in numeric id windows over SPARQL.

Source