What I learned putting macOS 27's on-device model behind a coding agent

Description and use

The author built fm‑mcp to reliably delegate tasks to Apple’s on‑device model, addressing context limits, runaway completions, and number parsing issues.

Problem

Developing an agent that delegates tasks to Apple’s on‑device model (fm) while avoiding common pitfalls such as context overflow, runaway answers, and mis‑parsed numbers.

Outcome

The agent now reliably delegates only tasks the model handles well, avoiding runaway completions, mis‑parsed numbers, and mis‑classified fields, while maintaining throughput and correctness.

Workflow

• 1. Measure token limits with fm count-tokens to determine safe input size.
• 2. Rewrite JSON schemas to include anyOf with title and unique names for nullable fields.
• 3. Use max_completion_tokens to cap answers and detect runaway completions.
• 4. Convert number fields from text to numeric, treating non‑numeric strings as null.
• 5. Preserve finished fields when a runaway occurs and add a note identifying the stuck field.
• 6. Add explicit label lists in prompts to improve classification accuracy.
• 7. Prefer plain vision requests over fm respond --tool ocr for text extraction from images.

Evidence

fm serve accepts max_tokens and ignores it. I asked for a 600-word essay with a cap of 30: with max_tokens the answers ran 587 to 699 tokens, with max_completion_tokens exactly 30.

Source

Structured output with a JSON Schema works, within limits: no $ref, no nullable type arrays like ["string", "null"], and pattern fails outright.

Source

On an invoice with four fields present and two missing, that came back exactly right in 5 runs out of 5, about 1.4 seconds each.

Source

Asked for the same amount as a string, the model copied it exactly every time it finished an answer, 46 times out of 46.

Source

On 200 news articles from the AG News dataset, with four topics, it got 49% right and almost never picked "Science and technology".

Source

One sentence in the prompt, "The labels are: …", took those to 64% and 58%.

Source