Frontier Model Performance Benchmark Runner

说明与用途

一段 Python 自动化脚本，用于在 Oxlo.ai 前沿模型上进行推理、代码生成和翻译任务的基准测试，并使用本地评审模型对结果进行评分，提供可复现的比较。

问题

Choosing the right flagship LLM for a product requires reproducible, task‑specific performance data rather than leaderboard screenshots.

结果

The script produces a table of scores per model per task, enabling quick identification of the best performer for each use case.

工作流

• Configure Oxlo.ai client and select models
• Define benchmark tasks (reasoning, code generation, translation)
• Collect answers from each model
• Define judge system prompt
• Score each answer with the judge
• Print a summary table
• Run the script and interpret results

证据

We loop over tasks and models, calling Oxlo.ai for each combination.

来源

We pass each candidate answer back to Oxlo.ai alongside the original task so the judge can assess correctness.

来源