Anthropic's Fable 5.2 Completes Over 41% of Real Freelance Jobs on Remote Labor Index

Anthropic's latest AI model, Fable 5.2, has demonstrated a major leap in practical capabilities by completing 41.3 percent of real-world freelance projects at professional quality, according to new benchmark results on the Remote Labor Index (RLI).

The RLI, developed jointly by Scale Labs and the Center for AI Safety (CAIS), measures how successfully autonomous AI agents can execute actual client jobs well enough to be accepted by paying customers. Fable 5.2's 41.3 percent score establishes a commanding lead, completing more than twice as many jobs as its direct predecessor and nearly doubling the performance of competing frontier models.

A Rapid Evolution in Practical Agent Work

The benchmark results highlight how quickly AI agents have progressed from isolated demos to executing sustained professional workflows. When the RLI benchmark launched roughly one year ago, the leading agent at the time—Manus—successfully finished just 2.5 percent of the tasks. Fable 5.2 now completes more than 16 times as many professional assignments.

The jump is similarly stark over shorter intervals:

• July: Fable 5 led the index at approximately 16 percent.
• Fable 5.1: Trailed far behind Fable 5.2, scoring just under 18 percent.
• Current Competitors: Anthropic's Opus 5.5 sits in second place at 21.25 percent, narrowly ahead of OpenAI's GPT 6 Astra at 20.83 percent. Google's top-performing system in the test, Gemini Argon 4, scored 17.08 percent, while older systems like GPT-5.2 and Gemini 3 Pro languish below 3 percent.

Breakthroughs and Current Limitations

According to Scale Labs, Fable 5.2 made its most substantial breakthroughs in audio and music editing, architectural and interior design, and 3D modeling. The system successfully completed nearly 50 complex tasks that no AI agent on the benchmark had ever solved before, demonstrating an ability to navigate long-horizon projects that typically cause models to derail.

However, significant barriers remain. The model still struggles with industrial product design and animation tasks. When agents fail on the RLI, researchers noted that they generally produce amateur-level outputs, broken project files, contradictory designs, or incomplete deliverables.

Testing Against $144,000 in Real-World Contracts

Unlike synthetic benchmarks that evaluate isolated coding puzzles or multiple-choice questions, the RLI evaluates end-to-end professional assignments. The evaluation suite consists of 240 completed projects sourced from 358 verified freelancers on Upwork, representing roughly $144,000 in actual paid work.

Each project in the test set costs an average of $632.60 and requires an average of just under 29 hours of human labor. Tasks span disciplines such as CAD drafting, video production, game development, and data analysis. To complete these assignments, AI agents operate inside virtual environments equipped with industry-standard software, including Blender, GIMP, and Audacity, supplemented by specialized AI generation tools.

Submissions are evaluated manually by trained human specialists who open the output files in the native software and compare them directly against the human freelancer's original deliverable. CAIS attempted to automate the grading process using evaluator AI models, but the automated judges failed; unable to proficiently navigate the specialized professional software, the automated reviewers were repeatedly misled by surface-level polish and awarded inflated scores to technically broken submissions.