How I Approach Reliable Web Scraping with Python

A reliable web scraping approach in Python: first look for an API, then use Scrapy, and only when necessary browser automation, with careful pagination and data