The thumbnail that lied
A capture tool that picked the earliest changed frame produced a duck game with no ducklings and a rhythm game showing MISSED. Fixing it meant letting each game declare when it is worth looking at.
Every thumbnail on the front page of this site is a real screenshot of the game running, captured automatically by a headless browser. Nothing is composed by hand, which means no thumbnail can advertise something the game does not actually do.
It also means a thumbnail can advertise something the game does not do yet, and that turns out to be the harder problem.
The timing question
Screenshot a game too early and you capture the title screen. Too late and you capture the game over overlay. Both are worthless as a thumbnail: one shows a button, the other shows a number and the word FAILED. The window in between is where the game actually looks like itself.
The first approach was to take the title screen as a baseline, then grab frames at 900, 1500, 2200, 3000 and 3800 milliseconds after triggering the start, and keep the earliest frame that differed from the baseline by more than a threshold. The reasoning holds up: a game that has started looks different from its title screen, and game over comes later than gameplay, so picking the earliest changed frame naturally avoids the end state.
It produced two bad thumbnails immediately, for opposite reasons.
Duck Line: the game without its game in it
Duck Line is about a line of ducklings trailing behind a mother duck. The longer the line, the worse it steers; the tail cuts every corner; the whole design is about the line.
At 900 milliseconds the water is already flowing and the scene passes the “this has changed” test comfortably. There is also exactly one duck on screen. The mother has not reached a single duckling yet. So the thumbnail — and the share card generated from it, which is what appears when anyone links to the page — showed a solitary duck in a pond, with no hint of the mechanic the game is named after.
The check was working exactly as written. It confirmed that pixels had changed, and pixels had. “Changed” is not the same as “legible”, and no pixel statistic knows the difference.
Pulse Lock: the failure feedback
The opposite failure showed up in Pulse Lock, a timing game. Nobody is playing during automated capture, so the game does what it should when a player misses every beat: it says so. By 1800 milliseconds a red MISSED banner covered the middle of the screen, and that is what went into the thumbnail.
Again the rule was satisfied. The frame was different from the title screen, it was the earliest such frame, and it was not technically game over. It was a picture of someone playing the game badly.
The fix: let the game say when it is worth looking at
Both failures come from the same source — the capture tool was trying to infer, from pixels alone, when a game becomes recognisable. That information is not in the pixels. It is in the design, and the only thing that knows it is the game.
So the game declares it. An optional captureFromMs field in the game metadata
moves the start of the sampling window, and the sampling continues to take five frames from
there. Duck Line sets it high enough that a few ducklings are in the line. The rest of the
catalogue leaves it unset and keeps the default window.
The important detail is what was not changed. The “earliest changed frame” rule still applies within the window, so the game-over protection that the rule was written for is intact. One number moves the window; the logic inside it is untouched. Fixing this by hand-picking a frame per game would have thrown away the protection along with the problem.
The same bug wearing a different hat
A related tool captures ten screenshots during play for review purposes, and it drives the game by pressing keys. Its first version alternated left and right.
In a snake game, pressing the opposite direction is suicide. All ten review screenshots of Cyber Snake were the game over screen — ten identical pictures of the moment the automated player killed itself, twice a second, forever. The input pattern is now a clockwise rotation through the four directions, which is never a reversal in any game and therefore never a self-inflicted death.
The tap position moved for a similar reason. Taps were landing in the lower middle of the screen, which is exactly where a game over overlay puts its “back to menu” button, so the capture tool kept clicking its way out of the games it was supposed to be photographing. Taps now land in the upper third.
What generalises
Three things come out of this that apply well beyond screenshot capture.
A proxy metric drifts from its intent under pressure. “Pixels changed” was a stand-in for “the game is showing itself”. It agreed with the intent in the common case and came apart in exactly the cases that mattered.
Automation makes a specific kind of mistake: the plausible one. None of these outputs were obviously broken. They were real screenshots of the real game at a real moment. They were simply the wrong moment, which is invisible until a person looks at the result and thinks “that is not what this game is”.
When the tool cannot know, let the subject declare. The same pattern shows up everywhere in this codebase — a game declares that it ends on a single action, or that it is never supposed to end at all, rather than the checker guessing from behaviour. Declarations are visible in source, reviewable, and impossible to fall into by accident. That idea has its own write-up in a gate that graded the opposite, where forgetting to wire one of these declarations cost two weeks.