Debugging Job Application Automation: The Cost of Trusting Broken Instruments

Description and use

A developer building a job application robot found production debugging expensive and slow due to unreliable local tests and telemetry. The solution involves manual page inspection, local logic debugging, and trusting only external confirmations from target systems rather than internal logs or screenshots.

Problem

Developing a browser agent for job applications is easy, but debugging it in production is expensive and slow. The core issue is that telemetry and local test rigs often lie or fail to capture real-world state, leading to wasted time fixing non-existent bugs while ignoring actual failures.

Outcome

The developer identified that most 'bugs' were actually failures in measurement instruments (telemetry, local rigs). By shifting to a workflow that prioritizes manual inspection and external verification over self-generated logs, they reduced debugging time and increased trust in the product's data.

Workflow

• Identify the failure in production (e.g., application not submitted).
• Avoid immediate redeployment; instead, inspect the live page manually like a human user.
• Reproduce the issue locally with a browser open to inspect DOM elements directly.
• Fix the code based on direct observation (e.g., correcting input type assumptions).
• Verify the fix locally before deploying to production.
• Deploy to production only to verify aspects that cannot be tested locally (capacity, anti-bot behavior, actual employer response).
• Trust only external confirmations (e.g., company reply) as evidence of success, not internal logs or screenshots.

Evidence

The actual bug... took two minutes. A field I was reading as a textarea was an input.

Source

An absence is not evidence until you have proved you were watching.

Source

The only thing that proves an application exists is the company's own system replying.

Source