Remote Labor Index puts Astra at 20.83% client-acceptable project completion

Description and use

<p>GPT-6 Astra achieved a 20.83% client-acceptable completion rate on Scale’s Remote Labor Index, while Fable 5.1 reached 17.92%. The result is a meaningful advance in end-to-end agent performance, but it also means that even the leading system failed roughly four out of five projects at the benchmark’s professional delivery standard.</p> <h3> Key facts </h3> <ul> <li>The <a href="https://labs.scale.com/leaderboard/rli" rel="noopener noreferrer">Scale RLI leaderboard</a> lists Astra at 20.83% and Fable 5.1 at 17.92%.</li> <li>RLI contains 240 freelance-style projects across 23 work categories;

Problem

<p>GPT-6 Astra achieved a 20.83% client-acceptable completion rate on Scale’s Remote Labor Index, while Fable 5.1 reached 17.92%. The result is a meaningful advance in end-to-end agent performance, but it also means that even the leading system failed roughly four out of five projects at the benchmark’s professional delivery standard.</p> <h3> Key facts </h3> <ul> <li>The <a href="https://labs.scale.com/leaderboard/rli" rel="noopener noreferrer">Scale RLI leaderboard</a> lists Astra at 20.83% and Fable 5.1 at 17.92%.</li> <li>RLI contains 240 freelance-style projects across 23 work categories;

Outcome

<p>GPT-6 Astra achieved a 20.83% client-acceptable completion rate on Scale’s Remote Labor Index, while Fable 5.1 reached 17.92%. The result is a meaningful advance in end-to-end agent performance, but it also means that even the leading system failed roughly four out of five projects at the benchmark’s professional delivery standard.</p> <h3> Key facts </h3> <ul> <li>The <a href="https://labs.scale.com/leaderboard/rli" rel="noopener noreferrer">Scale RLI leaderboard</a> lists Astra at 20.83% and Fable 5.1 at 17.92%.</li> <li>RLI contains 240 freelance-style projects across 23 work categories;

Evidence

Remote Labor Index puts Astra at 20.83% client-acceptable project completion

Source