When Simpler AI Decisions Worked Better: What I Changed in My Jev-Powered Game Judge

Описание и применение

Упрощение логики принятия решений в игре Sideways, где использовался Jev, привело к более точным и гибким ответам, улучшив взаимодействие игрока.

Задача

The AI‑powered Game Master in the Sideways puzzle game produced overly rigid responses, causing player queries to be rejected or answered poorly despite the model understanding the intent.

Результат

By simplifying the decision logic and reducing reliance on strict probability thresholds, the Game Master returned more accurate and helpful answers, improving player engagement.

Рабочий процесс

• 1. Build a semantic judgment model (Jev) that outputs probabilities for well‑formedness, relevance, evidence, etc.
• 2. Translate these probabilities into deterministic outcomes using TypeScript rules with 0.8 thresholds.
• 3. Observe that many player queries are rejected or answered poorly despite high model confidence.
• 4. Identify that hard gates and conservative thresholds cause loss of useful information.
• 5. Refactor the decision logic: lower thresholds, remove unnecessary hard gates, and simplify the mapping from probabilities to outcomes.
• 6. Deploy Version 3 and test with real player queries, noting improved response quality.

Доказательства

Making those semantic IF statements more sophisticated actually made the game worse.

Источник

The word mechines is obviously a typo. But the intended meaning is still easy for a human to recover.

Источник

0.7999 → rejected 0.8000 → accepted

Источник