48-Hour Field Notes: The Free Model Returned Valid JSON, and My Schema Said Nothing

Описание и применение

В ходе 48‑часового эксперимента обнаружено, что бесплатный уровень LLM возвращает пустые массивы, которые считаются валидными по схеме. Добавление minItems:1 и улучшение логики повторных попыток, а также предварительный тестовый набор позволяют стабилизировать nightly‑job.

Задача

A nightly extraction job using a free model tier returned empty arrays despite valid JSON, causing downstream reports to be empty.

Результат

Adding minItems:1 to the schema prevented empty arrays from being considered valid, reducing false positives. Refined retry logic and a pre‑flight probe mitigated free tier unreliability, leading to a stable nightly extraction job.

Рабочий процесс

• 1. Define JSON schema with minItems:1 for records array.
• 2. Run extraction job on free server with MonkeyCode free model.
• 3. Observe empty records and empty downstream report.
• 4. Add retry loop with backoff for transport failures.
• 5. Identify content failures and avoid retrying them.
• 6. Detect config errors (wrong model name) and stop run immediately.
• 7. Add pre‑flight probe with test cases for schema, ambiguous input, oversized input, wrong model name, idle window, and worker kill.
• 8. Verify probe passes before production run.
• 9. Deploy job with updated schema and retry logic.

Доказательства

My JSON Schema said "records": {"type": "array"}. An empty array is a valid array.

Источник

The one-keyword fix is "minItems": 1, and it is the single highest-value line in the whole article.

Источник

A mistyped model name produced a 4xx on every request. My retry wrapper treated everything that was not a 200 as transient, so the job cheerfully backed off and retried for hours.

Источник

Mid-run, the worker process disappeared and the journal I had been appending to went with it. The extraction calls were fine. The host was ephemer…

Источник