Your n8n Workflow Only Processed the First Page: Pagination Bugs That Look Like Success

说明与用途

n8n 工作流仅处理分页 API 的第一页，导致订单丢失但显示成功。通过检查是否存在更多页面、在每页后保存状态以及按业务键去重，解决了隐藏的数据丢失问题。

问题

An n8n workflow that fetched only the first page of a paginated API missed dozens of orders, causing incomplete reports without any error signals.

结果

The updated workflow reliably fetches all pages, resumes correctly after failures, and deduplicates records, preventing silent data loss.

工作流

• 1. Fetch first page of orders from API.
• 2. Write orders to spreadsheet.
• 3. Mark run as successful if HTTP response is received.
• 4. (Issue) No further pages fetched.
• 5. (Fix) Add loop to fetch pages until next cursor is empty or page size < limit.
• 6. Store cursor/offset after each page to enable resumption.
• 7. Deduplicate on order ID before writing.
• 8. Log counts of pages fetched, rows received, rows written, and duplicates skipped.

证据

The API returned 100 orders per page, and the workflow fetched exactly one page.

来源

My rule now: a fetch loop ends only on an explicit signal — an empty next cursor, or a page smaller than the limit — never on "we got one good response."

来源

Third, dedupe on the business key, not the row position.

来源