A year of AI coding in 701 commits: from "complete" to cause and verification

Before AI, right after, and a year later

The repository dates back to December 2020; the first trailer appears on August 7, 2025 (02cbb15).

Across the adoption line, length changed most. Before AI the median message was 26 characters, often one-liners like "issue fix"; right after, it was 233.5, about nine times longer. Causes did not come with the length.

We read all 114 fix commits (subject starts with fix, merges excluded) by hand. A message that says in a sentence why the symptom happened counts as clear; one that only hints at the cause through the change or a noun such as "caused by the banner" counts as ambiguous; one that lists changes only counts as none. August to December 2025: 1 clear, 6 ambiguous, 22 none. From September 2026: 56 clear, 11 ambiguous, 6 none. The single clear one in 2025 (fb1c8b4) has no trailer.

The keyword rule is a reproducible lower bound. It matches Korean words for "cause" and "because" and the past-progressive ending that describes a symptom ("the thing that kept failing"). Of its 33 hits after September 2026, 30 were clear by hand and 3 ambiguous; the 40 commits it missed held 26 more clear ones. A completion word is 완성 (complete), 완벽 (perfect) or 완전 (fully) in the subject; character counts are Korean characters.

On September 10, 2025, commit d605c37 was titled "test: unit tests 100% passing and code coverage complete". On September 13, 2026, commit 289ed55 wrote the symptom, the cause and the checks on separate lines (translated, excerpt):

fix(nav): regression where "press back again to exit" stopped working
…
The cause: the exit-confirmation logic lived inside an override of the deprecated Activity.onBackPressed().
…
Checked: 0 onBackPressed overrides, 0 direct calls, 0 direct KEYCODE_BACK handlers.
Verified: assembleDebug passes, all 174 unit tests pass.

The "Checked" line reports a search for the same defect on other code paths, with counts.

261 of the 326 trailer commits are from the last two months

Commits with a trailer numbered 8 in August 2025 and 31 in September 2025, then 157 in September 2026 and 104 in October up to the 19th. From January to August 2026 the repository saw only 20 commits in total.

All 21 completion subjects carry a bare "Claude" trailer with no model name. From September 2026 the trailers name Opus 5, Fable 5.1 and Opus 5.5, none of which appear earlier.

Two changes, a year apart

The first change: "complete" vanished after September 13, 2025. The 287 trailer commits since then contain no completion subject at all. We first blamed the instruction file. Claude Code reads the repository's CLAUDE.md every session, and by September 5, 2025 it had grown to 1,182 lines with the three completion words 73 times; the "perfectly identical" commit (b7eec61) lists "progress.md, CLAUDE.md: completion updated" among its changes. Yet the file still held 46 completion words in December 2025, and the 18 trailer commits from September 14 to December never used one. The file may have shaped the voice, but it cannot explain it alone. Why the words stopped after September 13 is not recorded.

The second change is fixes that state a cause: 2 of 7 were clear in February to April 2026, and 56 of 73 from September. Three first guesses failed here too.

• AI share: in September 2025, 31 of 34 non-merge commits (91%) already had a trailer, and that month had the most completion subjects, 16. In September 2026 the share was 99%, with no completion subjects.
• Rewriting the instruction file: commit 3ddf507 rewrote CLAUDE.md to 260 lines with no completion words on September 14, 2026. Before that, from September 1 to 13, 7 of 10 fixes already matched the cause rule, so the timing is wrong.
• Length: medians of 233.5 and 232 characters.

What remains is the model generation and the operator's own working rules. The repository does not record when the operator adopted a rule like "do not say done before it is verified", so the two cannot be separated.

Messages without a cause cost us, human commits included

1. August 9, 2025, 15:32: AI co-authored commit 5f082f8, "fixed the update notice problem", changed the app-version comparison from string to integer. Further down the same file, the content-version comparison stayed a string comparison.
2. Same day, 17:51: commit 71c31ae, with no trailer, put the string comparison back. Its whole message is "fix: 수정" ("fix: fix"). Why it was reverted is not written anywhere.
3. September 25, 2026: commit 7b997f8 finally fixed it and stated the cause. As strings, "6.7" sorts after "6.11", so content updates published from version 6.10 onward never reached existing users. The same commit also moved the app-version check that 71c31ae had reverted to a numeric comparison.

Counting from 5f082f8, the string comparison stayed for more than 13 months. 5f082f8 has no completion word, and 71c31ae was written by a person.

There is also a place where the agent was wrong and a person fixed it. On September 7, 2025, six minutes after an AI commit at 21:16, a person committed "restore the existing source that was wrongly modified (claude code... sigh)" (f43dada, 45 files, +596/−953). The "100% code coverage" in d605c37 is similar: no coverage tool (JaCoCo or Kover) was ever configured anywhere in the repository's history.

"100%" means something different in 2026. Commit e9cd771 on September 13 gives a denominator: of 26 errors reported by Naver (Korea's largest search portal, whose spell-checker the app calls), 16 were highlighted before the fix and 26 (100%) after. The commit that deleted a single R8 keep rule also wrote the cause and before-and-after numbers (Play Console DEX optimization and an R8 keep rule).

Do not count an agent's "complete" or "100%" in a commit as a progress signal. A fix commit without one line of cause and one line of how it was checked is not finished work.

Other approaches, and why we dropped them

• Trailer share: 91% and 99% produced opposite habits, so the share cannot separate them.
• Counting "regression" or Revert: commits mentioning a regression went from 0 in August to December 2025 to 9 after September 2026. The newer style names regressions, so this measure makes the recent period look worse.
• Classifying messages with an LLM: results shift with the model and prompt, so a reader cannot reproduce the numbers.
• A commit-msg hook or commitlint that requires a "Cause:" line on fixes: we did not use one. A hook checks that the line exists, so "Cause: unknown" passes, and adding it mid-measurement would blur the before and after. In a team repository, use the hook for the format and check the content with a hand-read sample each period.

What these numbers cannot tell you

We first wanted to write that regressions fell once causes were recorded, and dropped it because there is no data to measure that.

• 656 of the 701 commits (94%) come from one account. This is effectively a one-person repository; in a team, individual habits will mix in far more.
• The 206 merge commits are mostly branch merges from before July 2025, so commits inside branches survived. In a repository that squash-merges, this comparison will not work.
• A trailer means the commit was made with the agent, not a share of lines written, and AI-assisted commits without a trailer are invisible.
• The hand labels follow one reader's criterion. Wherever the ambiguous ones go, the gap between 1 of 29 and 56 of 73 remains.
• In October 2026, 54 of 101 non-merge commits were release and monitoring notes (docs) and only 11 were fixes, so that month's rates are noisy.

(Our first completion rule also matched "100%", "final" and "achieved". It caught 8 October subjects, 6 of them rollout percentages like "production 100%", so we removed those three words; the 21 in 2025 did not change.)

Running the same comparison on your repository

On October 23, 2026, changing only the dates, these commands reproduced 21, 0, 0 of 29 and 33 of 73 on this repository. The monthly and model commands include the excluded submodule commit, so April 2026 and Opus 4.6 come out one higher. --grep='^fix' also matches body lines; here it gave the same set as a subject-only filter. For an English log, swap the Korean words for your own, for example complete|perfect|fully and root cause|because|caused by.

1. Cut the periods first: before and after the AI tool, and around long gaps.
2. Fix the rule before running it, and log any change with the reason: changing words silently after seeing numbers gets you the conclusion you wanted.
3. Read the fix commits by hand: we read all 114 and sorted them into clear, ambiguous and none. With more, draw at least 20 at random per period.

# trailer commits per month (merges included)
git log --since=2025-08-01 --until=2026-10-20 \
  --format='%ad %(trailers:key=Co-authored-by,valueonly,separator=;)' \
  --date=format:%Y-%m | awk 'NF>1{print $1}' | sort | uniq -c

# subjects with a completion word
git log --no-merges --since=2025-08-01 --until=2026-01-01 --format=%s \
  | grep -cE '완성|완벽|완전'

# fix commits that state a cause
n=0; y=0
for h in $(git log --no-merges --since=2026-09-01 --until=2026-10-20 --format=%h --grep='^fix'); do
  n=$((n+1))
  git log -1 --format=%B "$h" | grep -qE '원인|때문|던 (것|문제|회귀|결함)|던$' && y=$((y+1))
done; echo "$y/$n"

# trailers by model name
git log --until=2026-10-20 --format='%(trailers:key=Co-authored-by,valueonly)' \
  | sed 's/ <.*//' | grep . | sort | uniq -c | sort -nr

What we check next

We plan to compare how often a file is fixed again within 30 days after a fix that stated a cause versus one that did not (measurement planned for November 2026). Only that number would let us say that better messages came with better outcomes. The app itself is on the SugarSpeller page.

FAQ

More AX notes

• Daily users rose after the redesign while organic D1 retention fell from 15.2% to 11.2%
• We built three photo apps before checking the market: this year's new apps had a median of 1,166 installs
• How we shipped four Android apps in 20 days with an AI coding agent
• Pre-gzipping Android assets, measured: zero APK savings, and .pack.gz became .pack with a FileNotFoundException
• A market:// link opens a store chooser on SKT-sold Samsung phones: set the Play package first
• A splash that never moved on: addOnCompleteListener(this) and onStop

All notes on Retrospectives and numbers · All AX notes