Decision models are English-first. I measured them in Japanese, then built one.

Description and use

A Japanese decision model, sokudan, achieves high choice accuracy and low score RPS, while English models suffer from a position prior bias. Cross‑language transfer preserves choice performance but not yes/no. A Japanese typed‑decisions benchmark with 0.946 back‑translation match was released.

Problem

Japanese users lack decision models that handle typed questions without output tokens, and existing English models exhibit a position prior that hurts ordinal questions.

Outcome

sokudan delivers high choice accuracy and low score RPS on Japanese messages, outperforming English models on ordinal questions. Position prior was identified and mitigated in Laya and other models. Japanese‑only training preserves choice performance in English but not yes/no.

Workflow

• 1. Generate 300 synthetic Japanese support messages (bench_ja).
• 2. Run laya‑multilingual on bench_ja to identify position prior issues.
• 3. Report findings via GitHub issues and pull requests to add presentation checks.
• 4. Build sokudan: fine‑tune modernbert‑ja‑310m on synthetic data, encode state and question jointly, add cumulative‑link head for score.
• 5. Evaluate sokudan on bench_ja and compare with Lev, laya‑multilingual, and majority baseline.
• 6. Test sokudan on English bench to assess cross‑language transfer.
• 7. Translate LocalLLaMA typed‑decisions into Japanese, back‑translate, and filter by match flag.
• 8. Publish sokudan weights on Hugging Face and provide pip install instructions.
• 9. Offer a server exposing /v1/systemone API for local use.

Evidence

laya-multilingual never picked the first level of an ordinal question in Japanese: 0 of 300. Same in English: 0 of 290.

Source

I built sokudan, an open 310M Japanese decision model. On 300 Japanese business messages: choice accuracy 0.880, score RPS 0.075, bool AUROC 0.844.

Source

Trained only on Japanese, its choice transfers to English almost intact; its yes / no does not.

Source

Machine translation needs checking, so every case was back‑translated and judged against the source by a second model. Overall match: 0.946.

Source