Stop letting LLMs guess your math: Why eval() is a death sentence for agents

Описание и применение

LLM не умеют точно выполнять математические расчёты; использование eval() приводит к ошибкам и уязвимостям. Решение – использовать Deterministic Math Expression Evaluator, который парсит выражения и вычисляет их безопасно.

Задача

LLMs predict characters, not perform precise calculations, leading to incorrect math results and security risks when using eval()/exec for evaluation.

Результат

Agents now compute math accurately and safely, with average latency ~855 ms, and eliminate RCE risk by avoiding eval()/exec() calls.

Рабочий процесс

• 1. Agent receives a math expression from user or higher-level task.
• 2. Agent calls the Deterministic Math Expression Evaluator tool with the expression string.
• 3. The tool performs lexical analysis to tokenize the input.
• 4. It parses tokens into an AST following strict mathematical grammar.
• 5. The AST is evaluated deterministically respecting operator precedence.
• 6. The tool returns the precise numerical result to the agent.
• 7. Agent uses the result to continue its task or respond to the user.

Доказательства

If you expose an LLM to a runtime via eval(), you aren't just giving it a calculator; you are handing over Remote Code Execution (RCE).

Источник