Text-to-SQL on Oracle: two prompt rules took a light model from 35% to 64%

说明与用途

在提示中添加两个 Oracle 规则后，Gemini 2.5 Flash 在 UniQL 上的首次尝试准确率从 35% 提升至 64.5%，错误数从 694 降至 85。

问题

Oracle’s reserved word syntax caused low first‑try accuracy of a light LLM on the UniQL benchmark.

结果

First‑try accuracy improved from 35.0% to 64.5% (≈15 points above best published model) by adding two Oracle rules to the prompt; failures dropped from 694 to 85.

工作流

• Run schemagate on UniQL Oracle questions with Gemini 2.5 Flash
• Detect Oracle target and inject two prompt rules: disallow "AS" before table alias and require quoting of reserved words "DATE" and "NUMBER"
• Execute queries, record first‑try accuracy
• Optionally run error‑repair loop for remaining failures

证据

Two mistakes caused almost all of it. Of the 694 failed first queries: 551 wrote FROM account AS a. Oracle doesn't allow AS before a table alias. 100 used a column named DATE or NUMBER without quotes. Both are reserved words.

来源

First-try accuracy went from 35.0% to 64.5%

来源