Fable 5.2’s Autonomous Builds: What the Demos Deliver, and What Still Needs Testing

1. Host and Subject Background

Matthew Berman is the principal speaker in this episode of Matthew Berman / Forward Future, a roughly 24-minute presentation centered on Fable 5.2. The episode combines hands-on demonstrations, descriptions of prompts and resource use, benchmark commentary and a personal assessment of the model. Its title’s “best model ever” framing reflects the position Berman argues throughout the presentation; the title itself supplies no independent ranking evidence.

There is no verified interview guest context here. Alex and Brian appear as members of Berman’s team, joining a multiplayer demonstration and contributing projects shown later. Their participation adds examples and interaction, rather than turning the episode into an interview about a guest’s background.

Berman’s relevant perspective is that of an enthusiastic tester presenting selected results. He says he has probably spent more than 100 billion tokens testing the model and calls it the best he has used. That self-reported investment helps explain his confidence, but the audience does not receive a complete test log. The useful subject of analysis is therefore the relationship between what he demonstrates, what he reports about execution, and the broader conclusions he draws.

2. What the Episode Covers

The opening game establishes the episode’s central contrast: a small amount of instruction can initiate a very large execution process. Berman plays a Gears of War-style level built in Unreal Engine, taking cover, using timed reloads and receiving a revival after being knocked down. He acknowledges that movement over a barrier looks awkward. His reported initial prompt was only three lines, asking for one convincing level. He then enabled UltraCode and let it continue. According to Berman, the project had already consumed 8.5 billion tokens, remained in progress and would represent about $15,000 at API pricing after days of work. “Basically one shotted” describes his interaction with the system, rather than a brief or inexpensive computation.

The productivity demonstrations shift the question from spectacle to recognizable workflows. A browser-based Excel clone reportedly took almost 50 hours. Berman opens formulas, loads fictional sales data and shows a pivot table, while describing the application as feature complete. After the sponsor segment, he displays a resource report containing approximately 8 billion cache-read input tokens and 15 million output tokens, with an estimated API cost of $11,700. Those categories matter: the report is more specific than a single aggregate token count.

Kestrel, a video editor running locally on Windows, supplies another set of concrete interactions. The presentation shows a waveform, splitting and moving clips, snapping adjacent clips together, keyboard shortcuts, zooms and keyframes. Berman also says Alex has been editing videos with a separately generated editor. That is a reported instance of internal use, rather than external customer validation or a full comparison with the application being imitated.

Tumble Beans provides a more direct test of shared interaction because Berman plays it with teammates. Collisions, bouncing, doors and disappearing floors become visible parts of the experience. He says the original request asked for ten fun, complex and different levels, with multiplayer added afterward. The reported execution was over 9 billion tokens and about 71 hours, at an estimated $13,000. The later multiplayer instruction is a meaningful part of the build history.

Smaller projects show a different resource scale. A water-balloon simulation offers replay, camera movement and adjustable parameters in a single HTML file using WebGL 2 without external libraries. Berman reports 41 million tokens, about two and a half hours and an estimated $91. A software-decompilation animation lasts 1 minute 52 seconds and reportedly required 9.5 million tokens, 45 minutes and about $25. Another explainer visualizes embeddings, transformer layers, training and next-token prediction. Together, these examples broaden the episode from software construction to interactive explanation without suggesting that every task carries the same budget.

3. Core Views: Reasoning, Examples, and Limits

The strongest underlying argument concerns how much supervision a complex task requires. Berman says the model made decisions on his behalf and estimates that nine times out of ten, or more, they matched what he would have chosen. The potential benefit is straightforward: when a system selects workable intermediate steps, the user can spend less time steering every detail. But agreement with one person’s preferences is a narrower standard than correctness. The episode supplies no decision inventory, scoring procedure or accounting of costly mistakes. His estimate describes an experience worth investigating, rather than a measured reliability rate. UltraCode’s use of multiple subagents also means the observed outcome belongs to a particular execution setup, with its associated expense.

Berman’s discussion of speed offers a useful evaluation principle. He reports code reviews finding bugs, security issues and small interface inconsistencies, while completing work much sooner than he expected. Tasks he previously thought might require 60, 90 or 120 minutes, he says, now take roughly ten minutes maximum. He explicitly distinguishes this from tokens generated per second: the benefit he perceives is a more direct route from assignment to completion. That distinction matters because a system can generate quickly while taking unnecessary steps, or generate less quickly while reaching an acceptable result sooner. Nevertheless, the older durations are partly expectations in his account. Without matched codebases, known defects and common completion criteria, the presentation does not establish a transferable speedup ratio.

The claim that “software is solved” requires a larger inferential leap. Selected spreadsheet and editing operations demonstrate that these artifacts have working behavior. They do not document every feature that Berman calls complete or flawless. The missing distinction is between producing a plausible application and establishing that it meets an explicit acceptance contract. The episode supports the possibility that increasingly complex prototypes can be developed with limited continuous direction. It leaves comprehensive compatibility, exception handling and ongoing maintenance to further investigation. Less effort spent directing construction does not by itself eliminate the work of deciding whether the result is dependable.

The visual demonstrations expose a related boundary. The balloon prompt asks for physical accuracy, and Berman says parameter changes recalculate the physics in real time. Neither statement supplies an accuracy measurement. Attractive deformation, reflections and adjustable controls can coexist with an unvalidated physical model. The Tokyo visualization similarly demonstrates an elaborate scene containing traffic, pedestrians, trains and reflections. Berman reports 6.72 billion tokens, about 70 hours and an estimated $11,000 for that project. Complexity and resource use establish neither complete geographic coverage nor fidelity to the real city.

The episode’s imperfections help calibrate its successes. Berman notices a tree in the house’s driveway, dismisses an event recreation as poor, and shows a person passing through others in the European town. These examples do not invalidate every generated artifact. They do show why overall visual coherence and local behavioral correctness deserve separate checks. A scene may be persuasive at a glance while failing precisely where a user needs navigation, interaction or a design decision to work.

Finally, Berman’s benchmark discussion contains its own qualification to the superlative framing. He reports Frontier SWE at 69.9, OS World 2.1 at 89.3 and Humanity’s Last Exam at 74.2. Yet he also shows Carbon leading Automation Bench, 53 to 51, and Mythos 5.2 leading Terminal Bench Science, 75.3 to 74.6. These remain episode-cited scores rather than independently verified results here. Even within the presentation, “best” expresses an overall preference across evidence, rather than first place in every task category.

4. Learning and Application

A practical way to use this episode is to document autonomous builds along two dimensions: requirements and resources. Save the original instruction, subsequent additions and acceptance criteria alongside runtime, human interventions and execution mode. For costs, preserve ordinary inputs, cache reads and outputs as separate categories whenever available. The spreadsheet report demonstrates why a large token total cannot simply be multiplied by one headline rate. Before expanding an open-ended build, run a bounded task with a stopping condition and inspect both the artifact and its resource consumption. That creates a basis for budgeting rather than extrapolating from how impressive a demonstration looks.

Model selection should then follow the work’s economics. Berman himself pairs his preference for Fable’s capability with praise for Opus 5.5’s speed, efficiency and lower expense. A useful local comparison would give candidate models the same real task and count generation, waiting, correction and acceptance work toward total completion time. The question becomes whether an improvement in usable quality justifies its additional cost. Berman quotes standard rates of $8 per million input tokens and $40 per million output tokens, versus $16 and $80 in fast mode. Those figures should remain attributed to this episode. His repeated claim of 2.5-times fast-mode speed was not based on his own access, so it should not become a delivery assumption without testing.

For interactive software, turn the intended workflow into repeatable acceptance exercises. A spreadsheet evaluation could specify the formulas, data volumes and file-exchange paths the user actually needs. An editing evaluation could follow an asset through import, cutting, keyframes, saving and final output. A multiplayer game needs explicit conditions for participant count, disconnection and sustained play. Tumble Beans supplies evidence of a functioning team play session. Berman’s description of it as sellable supplies neither sales results nor large-scale validation. Monetization planning should therefore include stability testing and user feedback as additional work, with time and budget allocated to them.

The printed tower illustrates why physical outputs require measurement beyond a successful-looking demonstration. Its stated constraints include PLA, a 0.4 mm nozzle, a target below 10 grams and fit within the printer. Berman reports approximately 10 million tokens, slightly over an hour and $37, then stacks devices and a drill on the result. To compare designs meaningfully, an experimenter would need the finished structure’s measured mass, measured loads, loading positions, failure behavior and repeated trials. The presentation does not provide the complete measurements needed for a dependable strength-to-weight result. It offers a prototype workflow, while leaving load-bearing suitability unestablished.

Finally, label evidence by its role. Eight Sleep is explicitly sponsor material; the closing discussion of recursive self-improvement speculates about future progress and internal models. Neither fills gaps in current performance validation. Keeping those categories separate makes the episode more useful: its demonstrations can motivate bounded experiments, while budgets, checkpoints and acceptance conditions determine whether reduced supervision produces a result worth adopting.

Source

• Original episode: Anthropic just dropped the best model ever... (Fable 5.2)