Text-to-SQL on Oracle: two prompt rules took a light model from 35% to 64%

Описание и применение

Добавление двух правил Oracle в подсказку повысило точность первого выполнения Gemini 2.5 Flash на UniQL с 35 % до 64,5 %, снизив количество ошибок с 694 до 85.

Задача

Oracle’s reserved word syntax caused low first‑try accuracy of a light LLM on the UniQL benchmark.

Результат

First‑try accuracy improved from 35.0% to 64.5% (≈15 points above best published model) by adding two Oracle rules to the prompt; failures dropped from 694 to 85.

Рабочий процесс

• Run schemagate on UniQL Oracle questions with Gemini 2.5 Flash
• Detect Oracle target and inject two prompt rules: disallow "AS" before table alias and require quoting of reserved words "DATE" and "NUMBER"
• Execute queries, record first‑try accuracy
• Optionally run error‑repair loop for remaining failures

Доказательства

Two mistakes caused almost all of it. Of the 694 failed first queries: 551 wrote FROM account AS a. Oracle doesn't allow AS before a table alias. 100 used a column named DATE or NUMBER without quotes. Both are reserved words.

Источник

First-try accuracy went from 35.0% to 64.5%

Источник