Two schedulers, one question: how we keep a 150-job small-business ops platform reliable

Description and use

Two schedulers keep a 150‑job small‑business ops platform reliable by separating infrastructure and business tasks, using a unified dashboard, and enforcing logging, retries, and staleness alerts.

Problem

Small‑business operations involve many repetitive tasks with hard deadlines (SMS confirmations, reminders, invoices, etc.). Missing any task can lead to client dissatisfaction.

Outcome

The platform now reliably executes 150+ scheduled jobs, with infrastructure tasks surviving app crashes and business jobs avoiding duplicate execution. Monitoring detects failures before clients notice.

Workflow

• 1. Classify each job as "needs to run when app is down" or "does not need to run when app is down".
• 2. Schedule infrastructure jobs (e.g., backups, watchdog) in the OS task scheduler (Windows Task Scheduler).
• 3. Schedule business jobs (e.g., reminders, reports) in the in‑app scheduler (Python schedule library).
• 4. Deploy a read‑only dashboard that lists both scheduler rings with last/next run times.
• 5. Ensure every job logs, retries, and reports failures.
• 6. Monitor staleness by checking if a job performed real work, not just exited successfully.
• 7. Use a watchdog to restart the app automatically, preventing duplicate job runs.
• 8. Send plain‑English alerts for routine successes or failures.

Evidence

Every recurring job lives in one of two places, and one question decides which: Does this job need to run when the app itself is not running?

Source

A backup job living inside the app does nothing on exactly the night the app has crashed, which is the night you needed it.

Source

A reminder job moved out of the app to "make it independent" loses the shared state and locks it was quietly relying on, and starts double‑sending.

Source