Inside the Lab · Field note 012 · Kicker
Five Billion Hands Later, Our Bots Had Taught Each Other to Shove
We trained a model on the wrong lesson. It learned it perfectly. Here is how we found out, and what retraining changed.
Kicker’s five house bots have played each other around the clock since October 8, about 2,000 hands a second. By early November that was more than 5 billion hands, each one chained to the last, each batch signed. The accounts balanced. The poker was terrible.
This is the follow-up to field note 001, where we fine-tuned a small decision model to turn a character’s written style into a playbook. We said then that the model had learned to reproduce our engine, and that if the engine had blind spots the model would inherit them. It did.
What we saw
In 65% of hands, somebody went all-in. Brann, written as the tightest character at the table, was winning 426 big blinds per 100 hands while entering 67% of pots. Odile was entering 93%. For scale, a strong human player is usually reckoned to win somewhere between 5 and 10.
We checked first whether we were counting wrong. We weren’t. Across the whole run, every bot’s results plus the rake came to exactly zero. The bots really were playing like that.
Finding the cause
Each character’s style is compiled once, by the model, into a table of 2,880 situations. Plain code then plays the table. So the fault had to be in one of three places: the engine, the model, or what we taught the model.
We built a small tool that seats the five characters and plays them straight from the training targets, with no model involved. It reproduced the live arena almost exactly: Brann at about +425, entering 67% of pots, an all-in in 65% of hands.
The model had learned its lesson faithfully. The lesson was wrong.
Why the lesson was wrong
The targets came from simulation: play each situation many times, measure what folding, calling and raising are worth, and turn those values into a strategy. We did that in two passes. The second pass measured values against our own loose first-draft characters, who called nearly everything.
Against players like that, the best play is to shove. So the targets learned to shove, the model learned the targets, and then every bot at the table was doing it to every other bot.
A second, smaller mistake: when we turned values into probabilities we softened them too much, which kept weak plays alive.
The fix: let it play itself until it settles
Instead of measuring against fixed opponents, each pass now measures against the previous pass’s own strategy, then moves halfway toward the new answer. We ran 15 passes.
| Pass | Hands simulated | Hands with an all-in | Best bot | Worst bot |
|---|---|---|---|---|
| Before | 65% | +425 | ||
| 1 | 3 million | 24% | +124 | −195 |
| 6 | 3 million | 13% | +69 | −139 |
| 9 | 60 million | 15% | +73 | −144 |
| 13 | 60 million | 7% | +31 | −70 |
| 15 | 60 million | 6% | +29 | −63 |
Win rates are in big blinds per 100 hands. It did not go down in a straight line. Passes 7 to 9 crept back up, and it only settled after we made the strategy less soft from pass 10 on.
The characters still differ, which is the point of having characters. In the final targets Brann enters 17% of pots and Pip 42%, and the loose ones lose to the tight ones at a believable rate.
Retraining
We built 168,000 examples from the new targets and made one pass over them, starting from the first model.
The job ran in the lab on a DGX Spark.
72.6 minutes
That covers 10,500 training steps on a single GPU, using 15 GB of its memory, while another model server kept working on the same machine.
We then asked both models the same 3,000 questions, in wording neither had seen, and compared their answers with the new targets.
| Picks the same top action as the target | 62.1% old model → 95.1% retrained |
|---|---|
| Distance from the target’s full split (0 is identical) | 0.280 → 0.040 |
| Folds seven-two to a raise | 45–94% → 80–98%, by character |
| Raises with aces facing a raise | 83–97% → 67–94%; the new targets call more often |
The old model’s 62% is not a score. It is the size of the change we asked for: in more than a third of situations, the right answer is now a different action.
The weakest group was characters described with both style dials and quirks, at 91.8%.
Before it went live
We compiled all five characters with the retrained model, 28,800 decisions, each with its own signed receipt, and played them against each other for 300,000 hands in the lab. Same seats and same seed as the run that had reproduced the problem.
The compiled characters played the targets to within the noise: an all-in in 6% of hands, Brann at +29 entering 17% of pots. Nothing was lost between the lesson and the table.
Live
On November 10 we installed the retrained model and recompiled the five house characters one at a time while the arena kept running. Once all five had switched, we read the next 1,120,000 hands from the arena’s hand files.
| Before, live | After, live | Lab prediction | |
|---|---|---|---|
| Hands with an all-in | 65% | 6.0% | 6% |
| Hands reaching a showdown | 14.9% | 15% | |
| Brann: win rate, pots entered | +426, 67% | +28, 17% | +29, 17% |
| Sable | +408, 73% | +22, 24% | +27, 24% |
| Mara | +147, 81% | +10, 28% | +6, 28% |
| Odile | −663, 93% | −13, 42% | −8, 42% |
| Pip | −331, 90% | −60, 42% | −67, 42% |
The live arena did what the lab said it would. Results plus rake still came to zero.
Limits
- Only about 444 of the 2,880 situations come up often enough to measure directly. The rest borrow from the nearest one that does.
- The table is coarse. Three of a kind, straights and flushes share one strength bucket, so the bots can’t tell them apart. One visible effect: they sometimes fold a set to a big river bet.
- Brann at +28 is still a lot. That is what a tight player makes against Pip, who is written to play too many hands and bluff too often.
- This is play money among five house bots. It says nothing about how they would do against people.
- Self-play settles on a strategy that is hard to exploit by itself. That is not the same as good poker.
What we took from it
The model was never the problem. Agreement with the targets was high the first time too. What we had not measured was whether the targets themselves described sensible play, and five billion hands of honest bookkeeping could not tell us that either. It took a tool that played the lesson without the model to see it.
So the check that now runs before any new model goes live is the plain one: seat the characters, play the targets, and look at the poker.
See it. Watch the characters play live, or visit the Kicker arena for the running numbers. Read field note 001 for how the model was built. Or talk to us about simulation, custom models and systems that keep their own evidence.