Blog
Claude Fable 5 shipped twice.Both versions play the same.
Sheil Iyer and Aneesh SachdevaGlia Intelligence
All postsIn June we published our first read on Claude Fable 5 and put it at the top of the poker table. Three weeks later we ran the whole benchmark again, same model, same shuffled decks, to see what it did the second time. It played almost exactly the same game.
The question behind this run was simple. If a model is going to sit at the top of a benchmark, we want to know how much of that result is the model and how much is the draw of the cards. So we replayed every game.
Fable's second run looks like its first. It raised at the same rate, played the same share of hands, bluffed about as often, and its written reasoning reads the same, down to the phrasing. Its appetite for risk did not shift.
Its win-loss record did move. Overall it won 52% of games the first time and 46% the second. Against GPT-5.5, the matchup we led with at launch, it went from 57% to 49%. Both changes are smaller than the swing you get from luck alone at these sample sizes, so we read them as noise, not a change in the model.
The first benchmark ran on June 9th and 10th. On July 2nd and 3rd we ran it again. Everything we control was held fixed: the same model, the same instruction to play to win, temperature set to zero, and the same random seed, which shuffles the deck into the exact same order both times. Ninety heads-up games against the same six opponents, card for card.
With the deck fixed, the only thing that can move the results is the model. If Fable made the same decisions, the two runs would match hand for hand. So any difference we see is a clean measurement of how much the model varies from one run to the next.
Across all 90 games, Fable won 47 the first time and 41 the second. As a win rate that is 52% against 46%, a gap of about seven points.
Seven points feels like something. It is not. A win rate measured over 90 games carries real uncertainty, because a handful of big pots can land either way. If you took an unchanged model and ran it twice, the two win rates could sit as far as 15 points apart, about nineteen times out of twenty, purely from the cards. Seven points is less than half of that.
The shaded band is how far two runs of the same model can land apart, about nineteen times out of twenty, from the draw of the cards. The gap we measured is less than half its width.
The pattern by opponent tells the same story. Against GPT-5.5, our launch headline, Fable went from 57% to 49%. Against Opus 4.8 it landed on 44% both times, exactly. The matchups that moved most were the small ones, where a single game is worth ten or twenty points and the number bounces around. The more games in a matchup, the less it moved.
The matchups that moved most were the ones with only five or ten games, where one game is worth ten or twenty points. Against GPT-5.5, the largest sample, the change was small. Against Opus 4.8 the win rate landed on 44% both times.
None of these differences is statistically significant. A test of the GPT-5.5 records returns p = 0.63; pooled across all 90 games, p = 0.46. In plain terms, we cannot tell the two runs apart from chance. The ±15 point figure is the 95% range for the difference between two runs of 90 games at an even win rate.
Win-loss records are the noisiest thing we measure, because they compress a whole game into a single result. The way a model plays is far steadier, because it is averaged over thousands of decisions. This is where the two runs are almost a photocopy.
Fable raised before the flop 38.4% of the time in June and 38.5% in July. Its overall aggression, the share of hands it played, and how often it bluffed all held within a point. The reasoning traces are the clearest tell: same length, same phrasing, same way of narrating a decision. Read side by side, you cannot tell which run a trace came from.
Aggression is the ratio of raises to calls, on the same scale used across PsychBench. Reasoning follow-through is the share of decisions where the written plan matched the action taken. Figures are averaged over all 90 games and roughly 2,000 decisions per run.
Here is one hand to make it concrete. Both runs are dealt king-nine on the button and reach the same decision. The reasoning is nearly word for word. The only difference is the size of the raise.
"K9 offsuit is a strong hand in heads-up play, well above average, and as the button I should be raising the majority of my range here. I'll make a standard raise."
"K9 offsuit is a strong hand in heads-up play, well above average, and as the button I should be raising a wide range here. A standard raise to about 2.5x the big blind is appropriate."
The same hand, dealt from the same deck, in both runs. Same read, same plan, a slightly different number. Small gaps like this are what set the two games on different paths.
One number did move in a consistent direction. The second run bet about 16% smaller on average. It is the only style figure that shifted across the matchups, and it is minor next to how much stayed put.
If the model is this consistent, why did any games flip? Two reasons, and both are ordinary. Temperature zero makes a model nearly deterministic, but not perfectly. Now and then it lands on a slightly different number, like raising to 1,500 instead of 1,800 on the same hand. And poker amplifies small changes. Every hand comes from the same deck, so the moment one bet is a little different, the play runs a little longer or shorter and the next cards come out in a different order. From there the two games drift apart.
Game one is the clean example. Both runs open with the identical hand, play it the same way, and then separate one hand later when the deck shifts by a card.
Fable wins the game and knocks out GPT-5.5 around the 48th hand.
Fable is the one eliminated, on the 30th hand.
Both games open with the identical hand: Fable is dealt the king and five of clubs, raises, and GPT folds. One hand later the deck has shifted by a card, the two games start seeing different cards, and they finish in opposite places.
The takeaway is not really about poker. It is about how to read any single benchmark number. When a model tops a leaderboard by a few points, that margin can be the model or it can be the draw. The only way to tell them apart is to run it more than once and know how wide the normal swing is.
Our first Fable run put it narrowly ahead at the top of the table. This run tells us the lead is real as a range rather than a decimal, and that the model behind it is steady from one day to the next.
For anyone putting a model into production, that is the useful habit. Before you trust a number, ask how much it moves when nothing changes. A model that behaves the same way twice is one you can plan around. Fable did.