Puzzle game day-one churn: per-device event timelines, exact board math and four walls in chapter one

Three of ten boards could not be won by a line-first player

A "3-move turn" puzzle gives you a fixed board and three pieces; you must place all three and clear at least the target number of lines. For each board we played out every order and placement and computed exact win probabilities for two player models, with no sampling. "Line-first" picks among line-clearing moves when any exist, and among all moves otherwise. "After hint" fixes the first move to the correct one and plays the rest line-first.

R006 and R013 stay at 0% even when you are told the first move. Both need you to leave a clearable line alone on the second-to-last move and clear more at once with the last piece. R012 is 25% line-first, but with one winning sequence in 198 it went out with the others. R008 and R009 are also low line-first, but reach 100% after the hint, so they were treated as hint-solvable.

We lined up 16 devices' events by minute

A GA4 funnel only gives head counts per step, so we pulled one more Data API report: dimensions deviceModel, dateHourMinute and eventName, metric eventCount, filtered to country South Korea and firstUserMedium = cpc. Drop user_engagement, screen_view and session_start, group the rows by device, sort by time, and each player's day reads as one line. This is the request body we sent on the morning of 10-17 (property id removed).

{
  "dateRanges": [{"startDate": "2026-10-11", "endDate": "today"}],
  "dimensions": [{"name": "deviceModel"}, {"name": "dateHourMinute"}, {"name": "eventName"}],
  "metrics": [{"name": "eventCount"}],
  "dimensionFilter": {"andGroup": {"expressions": [
    {"filter": {"fieldName": "country", "stringFilter": {"value": "South Korea"}}},
    {"filter": {"fieldName": "firstUserMedium", "stringFilter": {"value": "cpc"}}},
    {"notExpression": {"filter": {"fieldName": "eventName",
      "inListFilter": {"values": ["user_engagement", "screen_view", "session_start"]}}}}
  ]}},
  "limit": 2000
}

If no level_up follows tutorial_complete, that player solved zero boards at R004. Seven devices matched, and five of those timelines also carry an app_remove. Banners and interstitials start only after three finished non-tutorial boards (the constant BANNER_FROM_RUN = 3). Those seven never finished one, so they met neither. That does not prove they saw no ad at all: the app-open ad on return follows other rules, and for 4 of the 7 we only read a table with ad impressions filtered out.

The timelines also showed why the averages misled us. Of 7,471 seconds of engagement across 16 devices, 4,505 (60%) came from a single device. The median was 148 seconds. That device was also the only one with a record past R013.

Both the ad theory and the first-move hint missed

On 2026-10-15 an aggregate report written by an AI coding agent read the Korean paid installs as "people who played plenty and then deleted," and suspected ad density. It leaned on the average playtime per user and on one device with many ad impressions. Two days later the operator asked for a deeper analysis, and the per-player rebuild came out of that request. The player who pulled the average up was the one who got furthest, and the ones who left were gone before any banner or interstitial could show.

The first fix was wrong too. The agent framed the churn as a cliff between tutorials with a dotted guide path and boards without one, and built 1.1.2: on R004 to R013, a free first-move hint after one fail and the solution after two (commit d1f6de1). At 08:54 on the morning of 10-17, in the same minute that build went up to Google Play, the exact per-board computation arrived. R006 and R013 stayed at 0% even with the hint. Thirty-four minutes later 1.1.3 (commit cc4e860) went up with a new board order and replaced 1.1.2.

The record overstated things too. The commit message and the analysis document call all four boards "0% line-first," but recomputed, R012 is 25% and R011 reaches 67% after the hint.

Why a board tagged "easy" became a wall

The four walls sat in chapter one because the pack was ordered by tier label. And the tier was assigned by the share of opening moves that can still win.

That share knows nothing about how people play. R006 has 2 winning openings out of 10 (20%) and was tagged easy, yet line-first wins 0% of the time. R010 has 1 out of 11 and was tagged medium, yet line-first wins every time. A random player's win rate did not separate them either: the nearly empty first tutorial R001 scores 2.58%, and R004 and R006 both score 2.75%.

R004 is only half explained. It is not a wall: 33% line-first, but 100% after the first-move hint. Still, it was the first plain board right after three tutorials with a dotted guide (59% to 99% of their openings can win), and only 4 of its 13 openings win. The new order puts R294, 100% line-first, at position 4, where those 7 devices stopped, and pushes R004 to position 11. With no fail events at the time, we cannot tell whether they got stuck on R004 or left without really trying it. The four walls are a finding about the chapter, not proof of why those 7 left.

We changed only the order and kept progress keyed by puzzle id

No board was regenerated. Progress is saved by puzzle id while navigation follows list position, so reordering the loaded list leaves saves intact. PLAY_ORDER puts boards that the line-first model wins easily right after the three tutorials and parks the four walls at positions 37 to 40. PlayOrderTest checks for lost or duplicated boards and pins positions 37 to 40. The new fourth board has pack id R294, so the on-screen number switched to the play position.

Across the ten boards at positions 4 to 13, the computed line-first average goes from 29% to 89%, and the average after a first-move hint from 68% to 98%. These are model numbers, not measurements of people.

Instrumentation shipped with it: level_start, level_fail with the fail count and a stuck flag, hint_free and solution_shown, each tagged with the puzzle id (pid) as well as the position. The release (R8) build passed the end-to-end run on a real device; failing the fourth board on purpose brought up the free hint and logged the pid. Monkey runs, 3 seeds x 800 events, gave 0 crashes and 0 ANRs.

Why we reread players one by one instead of the funnel

The averaged funnel had already given one wrong answer; with 16 devices, one device sets the average. Session replay and surveys were not considered that day; in our judgement neither helps with players who are already gone.

On the difficulty side, the alternatives were a random player and a greedy bot. The random win rate could not tell R001 from R004, and the greedy bot's result on some boards changed with its tie-breaking rule. A bot that plans the whole board was what we used to tune classic mode in Block puzzle difficulty tuned with simulation bots, but chapter one is about the habits of someone who does not plan, so the line-first model fit.

Do not order a first chapter by tier labels or random win rates. Order it by exact per-board numbers from a model that plays like a person.

When this method does not hold

• We grouped by device model, not by person. Two people on the same model merge into one line, and a reinstall shows up twice. The period had 19 first opens on 16 device models, which the analysis counted as 17 players.
• The board-number parameter was not registered as a custom dimension, so stopping points were counted from level_up events. Some devices logged four of them for three tutorial boards, so a position can be off by one. Custom dimensions only collect data from the day you register them.
• Line-first is a model. Sixteen devices cannot validate per-board churn; the real failure distribution will only show once level_fail data builds up.
• With more installs, players start to share device models. In our judgement, from a few hundred installs a day the device model can no longer separate people, and a raw event export that keeps a per-user identifier is the right tool.

Steps to repeat this

1. Filter to one acquisition slice (country plus first-user medium) and exclude your test device models.
2. Pull eventCount by deviceModel x dateHourMinute x eventName, minus the three automatic events. Group by device and sort by time.
3. For each player, write down the last progress event, whether they uninstalled, and whether they saw an ad before that, all in one table.
4. Compute the content where they stopped exactly, with a model that plays like a person. Treat random win rates and tier labels as hints only. Our line-first probability is the recursive mean below, memoized on board state. In one Python 3.12 process, line-first and after-hint probabilities for all 300 boards took 0.4 s, and enumerating every sequence took 2.3 s.

   win(board, left, lines):
     if left is empty: return 1 if lines >= target else 0
     moves = every legal placement of every piece kind in left
     if moves is empty: return 0            # cannot place: lose
     clear = moves that clear at least one line
     pool = clear if clear else moves
     return mean(win(board after move, left - piece, lines + cleared) for move in pool)
   
   line_first = win(start board, 3 pieces, 0)
   after_hint = win(board after the correct first move, 2 pieces left, lines it cleared)

5. Before the next release, add start, fail (with a stuck flag) and help-used events, tag them with a content id separate from position, and register those custom dimensions the same day.
6. Before reordering, confirm saves are keyed by content id rather than position, and pin no-loss, no-duplicate and fixed-position rules in a unit test.

Added 2026-10-24: the retest missed two of three bars

On 2026-10-24 we judged the first seven days after 1.1.3 (2026-10-17 to 10-23) of Korean paid installs: 25 installs, with test devices and emulators excluded. Only one of the three bars passed, same-day uninstalls.

Same-day uninstalls fell, but counting everyone who uninstalled within seven days gives 10 of 25 (40%). The sample is small, too. Sixteen people started position 4, so one person moves that rate by about 6 points.

The bar was named "R004 solve rate", but we judged it at position 4. Since 1.1.3 that slot holds R294, and R004 sits at position 11 (PLAY_ORDER in cc4e860). So the 10-of-17 baseline belongs to R004 when it was the fourth board, and the 62% belongs to R294 in the same slot. Versions 1.1.4 and 1.1.5 shipped during the window, and neither touched the play order; 1.1.5 appended R301 to R600 and left R001 to R300 as they were.

Players drop out in two places. The first is the opening tutorial R001: 21 started and only 16 solved it, a stretch we could not see in the baseline window (10-11 to 10-16) because there was no level_start event. The second is position 4.

The body said the four walls did not prove why those 7 devices left. This result settles that part. With the walls parked at positions 37 to 40, 6 of 16 players still failed to solve position 4, and swapping R004 for R294 left the solved count at the same 10 as the baseline. The baseline grouped players by device model and inferred progress from level_up counts while this run counts users tagged with pid, so we do not read a one-person gap as better or worse.

The model missed here as well. Line-first computed R294 at 100%; people solved it 62% of the time. It found the 0% boards in chapter one, but it did not predict how people do on the first board after the tutorials. Judge the first plain board by the share of people who solve it, not by a model number.

The second number we promised is not in yet. The verdict funnel stops at position 6 (R065, 7 starts), and level_fail for R004, now at position 11, and for the four boards at 37 to 40 was not tallied per board. The analysis record for the same stretch (10-17 to 10-22) does show one Korean paid player who solved R001 to R282 by pack id, so at least one person got past R004 and all four walls.

What we check next

The next fix candidate is the board at position 4. How many players actually stall on R004 (now position 11) and on the four boards at positions 37 to 40, by the pid on level_fail, is not tallied yet. The game itself is on the Block Turn page.

FAQ

More AX notes

• A reverse-construction generator and the gate behind 1,100 arrow-puzzle levels
• Testers said the time was "too long, so it's easy": checking that with a simulation before cutting puzzle par times by 30%
• Block puzzle difficulty: how simulation bots caught a game you could never lose
• Pre-gzipping Android assets, measured: zero APK savings, and .pack.gz became .pack with a FileNotFoundException
• A market:// link opens a store chooser on SKT-sold Samsung phones: set the Play package first
• A splash that never moved on: addOnCompleteListener(this) and onStop

All notes on Game development · All AX notes