48-Hour Field Notes: The Free Model Returned Valid JSON, and My Schema Said Nothing

Description and use

During a 48‑hour experiment it was found that a free LLM tier returns empty arrays considered valid by the schema. Adding minItems:1 and improving retry logic, plus a pre‑flight test set, stabilizes the nightly job.

Problem

A nightly extraction job using a free model tier returned empty arrays despite valid JSON, causing downstream reports to be empty.

Outcome

Adding minItems:1 to the schema prevented empty arrays from being considered valid, reducing false positives. Refined retry logic and a pre‑flight probe mitigated free tier unreliability, leading to a stable nightly extraction job.

Workflow

• 1. Define JSON schema with minItems:1 for records array.
• 2. Run extraction job on free server with MonkeyCode free model.
• 3. Observe empty records and empty downstream report.
• 4. Add retry loop with backoff for transport failures.
• 5. Identify content failures and avoid retrying them.
• 6. Detect config errors (wrong model name) and stop run immediately.
• 7. Add pre‑flight probe with test cases for schema, ambiguous input, oversized input, wrong model name, idle window, and worker kill.
• 8. Verify probe passes before production run.
• 9. Deploy job with updated schema and retry logic.

Evidence

My JSON Schema said "records": {"type": "array"}. An empty array is a valid array.

Source

The one-keyword fix is "minItems": 1, and it is the single highest-value line in the whole article.

Source

A mistyped model name produced a 4xx on every request. My retry wrapper treated everything that was not a 200 as transient, so the job cheerfully backed off and retried for hours.

Source

Mid-run, the worker process disappeared and the journal I had been appending to went with it. The extraction calls were fine. The host was ephemer…

Source