AI coding agents and secrets: one rule in chat, logs and token permissions, and the two exceptions a copied masking regex missed

One rule, three places it has to hold

The rule fits in a sentence: never copy a secret's value anywhere; say where it is and what kind it is. With a coding agent in the loop it has to hold in three places. Whatever the agent reads ends up in the session log, the code it writes can put values into logs and exception messages, and the deployments it designs can ask for broader tokens than the code needs.

In all three places the agent did the fixing. The person running these projects (the operator below) made two main decisions: the rule should apply to every session, and the agent should stop handing work back and do it itself.

Chat: the first secrets rule was written 47 seconds after a key reached the session log

The agent printed the last 150 lines of the shell history at 8:47 on October 2. The operator had asked it to fill in the shared instructions every session reads, and it wanted to learn the setup. One of them exported an AI API key. Tool output is saved verbatim in the session log, so the key was now part of the transcript.

At that moment the instruction file held only rules about reply format. Forty-seven seconds after the key was printed, the agent wrote the first secrets rule into it: one line saying secrets stay out of any output. It said what not to do and nothing about how to check, and for an agent, reading a file and printing it are the same action. Eighteen seconds later the agent told the operator that an AI API key sat in plain text in the shell history, giving the location and kind but not the value.

At 8:55 the operator asked for a rule that keeps passwords and personal data out of every session's chat, and the agent replaced the one line with a rule that also says how to check. The gist: never write a value, only where it is and what kind. Files are checked with commands that show key names and line counts (grep -c, cut -d= -f1), and commands take environment variables or file references instead of values. It covers commits, docs and memory files as well as chat, and a value someone pastes in is not quoted back; the agent recommends rotating it. The scope also widened from secrets to personal data.

The agent deleted the four history lines that held the key (three export lines and one secret-registration command that took the value as an argument) and counted again: zero. It reported two places it left for later: the log file of the session still running, which could break if edited mid-session, and the terminal window where the key had been typed. The report ended by saying that deleting doesn't undo the exposure and rotating the key is the only sure fix, and that a new key should be registered by name with the value typed at the prompt.

On October 25 we recounted across the 109 main session logs. The key-names-only command cut -d= -f1 appears 24 times, every dated one from October 3 on; with subagent logs included it is 2 in September and 179 in October. Counting .env lines with grep -c was already a habit in September (25 times), so what grew is the names-only check. Lines matching three known key shapes (the newer AI API key format, GitHub tokens, sk- keys) came to exactly one across all 2,567 logs including subagents: that 8:47 output.

# .env: line count and key names, no values
grep -c '=' .env
cut -d= -f1 .env

# shell history: how many lines mention that key name
grep -c 'API_KEY' ~/.bash_history

# commands reference values instead of containing them
curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com/v1/ping
wrangler secret put SIGNING_KEY < secrets/signing-key

Logs: four copies of a masking regex, and two exceptions that never masked

The chat app (Flutter) passes a shared token as ?token= in the signaling server URL. On September 7, the agent reviewing platform security before release rated this High: the discovery logs went through debugPrint, which prints in release builds too, and four connect-log lines wrote the whole URL.

The reviewing agent proposed Uri.parse(url).replace(query: null). The agent assigned to fix it reported that this keeps the query: in Dart's Uri.replace, a null argument means "keep the existing component". It switched to a one-line regex instead.

The copies came from how the work was split. Several fix agents each owned a set of files that day. The mobile owner's scope was "one masking line" in its two files, and the two web files went to another owner as a request. Shortly after midnight, the web side got its own _maskToken wrapping the same regex. A docs pass on September 8 and 9 wrote down "regex duplicated in 4 files; extracting a shared helper is still open", while the defect list already said Fixed, dated September 7.

On September 13 (555680e), writing the shared function meant looking at every place each file turned the URL into a string. Two HttpException sites in the mobile HTTP client had no masking at all: a failed SSE connection put the token into the exception's uri, and a failed POST put it into both the message and the uri. The mobile owner's first fix had covered the success log line only, and the failure exceptions in the same file were outside it. The web copies masked their failure messages too.

String maskSignalingUrl(String url) =>
    url.replaceAllMapped(_signalingToken, (m) => '${m[1]}***');

final RegExp _signalingToken = RegExp(
  r'([?&](?:token|secret)=)[^&#\s]*',
  caseSensitive: false,
);

The new pattern ignores case, also covers secret=, and stops at # or whitespace. We ran the old pattern ([?&]token=)[^&]* and the new one through Python's re on placeholder values to see the difference:

Three kinds of tests went in with it: 7 rule tests, 1 source guard, and 1 real failure path. The failure-path test starts an HTTP server on the loopback address that answers 401 and checks that the client's exception text contains token=*** and not the value. The source guard, shortened:

test('no copy of the masking regex comes back in discovery', () {
  final offenders = <String>[];
  for (final f in Directory('lib/core/discovery').listSync()) {
    if (f is! File || !f.path.endsWith('.dart')) continue;
    if (f.path.endsWith('discovery_log.dart')) continue; // the definition
    final src = f.readAsStringSync();
    if (src.contains('token=)') || src.contains('_maskToken')) {
      offenders.add(f.path);
    }
  }
  expect(offenders, isEmpty);
});

It fails if the regex fragment token=) or the old helper's name shows up in any file other than the definition. The duplication stayed for five days after it was written down on September 8, so copies are now blocked by a test rather than left to review. Keep one masking function, and commit a test that fails when a copy appears.

The same commit replaced the peer's public-key fingerprint in the diagnostics map with a yes/no flag. That map goes straight to the diagnostics screen, reports and logs; the fingerprint is meant for a person to compare by eye, so the confirmation dialog reads it on its own.

Release logcat also carried what users typed

On October 17, QA on the SugarSpeller spell checker found the same class of bug in its release logcat (21895f6). Up to version 1.6.030, release builds logged the check request URL (which contains the user's text), the first 500 characters of the raw response, a session key for the external checking service on every resume, and the FCM push token. In the app's logging wrapper, log.e prints regardless of BuildConfig.DEBUG.

The fix followed the chat app: diagnostic lines moved to log.d, and keys and text are logged only as a length or a present/absent flag. ReleaseLogSecretsTest scans all main sources for log calls that print in release and looks only at interpolations and arguments outside string literals. Its first test feeds in the 7 old leaking lines and requires all 7 to be caught; the test comment says that if these are missed, the scan below is fake too. A release build of 1.6.031, the next development version, went onto a Galaxy S20, and logcat showed zero such lines. The same yardstick first appeared in 3,066 green tests hid a line that breaks opening the database: checking tests by deleting the guard.

Permissions: the Worker was redesigned instead of adding permissions to the token

The family app keeps photos in Cloudflare R2, with a media Worker handling uploads and reads. The deploy notes from the afternoon of October 10 listed three jobs for a person: add Workers KV edit and billing read to the deploy token, create a separate token for the usage analytics API, and issue an R2 S3 key pair so Firebase functions could delete objects. The reason was a free-tier guard that read Cloudflare's analytics API every hour and wrote a pause flag to KV.

At 22:52 that night the operator told the agent to do the work itself instead of handing it over. At 23:13 the agent reported that work to make the Worker run without any extra token was already under way, and at 00:04 it committed 2c906c7: 33 files, 1,238 lines added and 1,146 removed. The first design, one new credential per feature, had been the agent's own; this time it rebuilt the same features on the R2 binding, the Durable Object and the shared key it already had. By design the token needs 2 permissions: Workers Scripts Edit and R2 Edit.

• A Durable Object (a stateful object that runs next to the Worker) now counts the day's Worker requests, Durable Object requests and R2 reads and writes itself. When the total passes 25 to 50% of the free allowance (the share moves with the time of day), uploads and reads close for the day and reopen at the next midnight, Korea time. The 248-line cron file that read the analytics API is gone.
• Functions delete R2 objects through a signed DELETE request to the Worker instead of S3 credentials. The signing key is derived from an HMAC key the functions and the Worker already shared; the signature covers method, path, keys, timestamp and nonce. Timestamps must be within ±60 seconds, a nonce works once, there is a daily cap of 3,000 requests and 30,000 objects, and only paths under the requesting family's folder are deleted.
• Worker secrets went from 3 to 1, and billing read was never added. Checking once in the dashboard that the account is on the free plan was left to the operator.

The cost is written down in the ops notes. Because the delete-signing key comes from the shared HMAC key, all 4 function service accounts that can read that key could in theory sign a delete. Before, only the delete function's account could read the R2 token. Narrowing it again means a separate delete key, which is one more Worker secret.

A read-only ops check on October 11 looked at the real token and found 3 permissions, one more than the design: Workers Scripts Edit, R2 Edit and Account Settings Read. With no billing read, the plan API answered with auth error 10000, which also confirms this token cannot change the plan or billing.

A copy written another way passes the source guard, and masking does not protect the token

The source guard looks for two strings in one folder. A regex written another way, or a copy in another folder, passes. The spell checker's log scan works line by line, so a log call split across lines gets through.

Masking does not protect the secret itself. The chat app's token is a constant in the app binary and can be extracted, which is why the app treats it only as spam control. The server also accepts the token in a header, but the client still sends it in the query; the review found the web client cannot set that header, so the fix closed the path from URL to log instead.

The chat rule works only while the agent follows it, and the count only finds key shapes we know.

The same night the Worker was reworked to need no extra token, the agent's own tool permissions grew. The agent's permission checker blocked the IAM grant commands, and approval in chat did not lift the block; an allow rule in the settings was required. The agent asked for two lines, one script and all gcloud commands, and the operator replied four minutes later that they were added. At 23:13 it asked for three more lines allowing the workflow tool and all npx firebase-tools and curl commands. How we scope what an agent may do on its own is in Running an AI marketer on a schedule four times a day: the permission table that leaves money, accounts and policy to a person.

Checks to add to agent instructions, masking code and token scopes

1. Put checking commands next to the prohibition in your agent instructions: grep -c, cut -d= -f1, and $NAME references in commands (on the file itself: How to write AGENTS.md: only the rules an AI coding agent gets wrong).
2. Don't let the agent read the shell history or .env whole. Have it count the lines that mention a key name; delete any key it finds there and rotate it. Pass new values through a prompt or a file, not a command argument.
3. For each kind of string that carries a secret (a URL, say), keep one masking function and call it everywhere that string is built: logs, exception messages, exception uri fields, diagnostics maps.
4. Write three tests: the rules (case, other key names, fragments, the same word in a path), a source guard (regex fragments outside the definition file), and a real failure path (a 4xx from a local server).
5. Feed a log checker the old leaking lines first and confirm it catches them.
6. Install a release build on a real device and search logcat by key name or marker. Grepping for the value puts the value back into your shell history.
7. For tokens, write down the permissions each feature needs, and before adding a credential check whether an existing binding or key can do the job. Confirm the real scope with read-only calls.
8. Where something got wider, write one line: what widened, and what it would take to narrow it.

# release logcat: count token= lines that are not masked
adb logcat -d | grep 'token=' | grep -vc 'token=\*\*\*'

One stale doc line, one transcript line, one undeployed Worker

The chat app's security doc still says "regex duplicated in 4 files" as of its September 16 commit; the code changed and that line did not. The key-shaped line in the October 2 session log has not been deleted, and the logs don't show whether that key was rotated. For the family app, once the Worker is deployed, we need to confirm uploads and deletes work with the 2 designed permissions and whether the extra Account Settings Read is needed at all.

FAQ

More AX notes

• A shared LLM cache could have frozen a prompt injection: the blind A/B that let us stop sending context
• What "100% E2E pass" really meant: deleting seven runners with zero expect() calls
• We built an edit feature and shipped it switched off: the HWP save gate that caught a form-field bug
• Our contrast checker passed while skipping 40% of the text: make it print how much it measured
• Pre-gzipping Android assets, measured: zero APK savings, and .pack.gz became .pack with a FileNotFoundException
• A market:// link opens a store chooser on SKT-sold Samsung phones: set the Play package first

All notes on Testing and release · All AX notes