The Redaction Log That Leaked More Than It Removed
Compliant by Design — Essay 06
Before I made a repository public, I ran a scan for leaked secrets. It came back clean, and I had a positive control to prove the scanner actually worked. Then a second sweep, the one that looks for names rather than credentials, found the largest disclosure in the whole repo. It was sitting in the one file whose entire job was to record what I had already removed.
What I was doing
The repo was a small open-source pack of Australian-compliance skills and checks. I had kept it private while I built it, and I was about to open it up. Two files do the actual work. Everything else is scaffolding: a changelog, a licence, notes to myself. Seven commits, one branch.
Before flipping it public I worked through a gate I had written for exactly this moment. Two of its blocks matter here. The first is a secret scan. The second looks for names.
Why the scan was green, and honestly so
The secret scan covered the full history, every commit, not just the current files, because history is exactly what becomes public the moment you flip the switch. Before trusting a clean result I ran a positive control: I committed a fabricated, credential-shaped string to a throwaway clone and confirmed the scanner flagged it. One finding, as expected. An earlier format I had tried was not detected at all, which is its own lesson about trusting a tool you have not tested against a known-bad input.
Then the real scan, against a complete mirror. Zero leaks. The credential check passed, and it passed for real.
The sweep that found what the scanner could not
The second block looked for names, not secrets: product names for things I have not launched, a company name I had decided months earlier would never appear in anything I publish, contact details. A rules-based secret scanner does not care about any of that. None of it is shaped like a credential.
Months earlier I had removed three unreleased product names from a single line of one skill file, in an ordinary commit. I expected that to surface in history. The original line still sat in the first commit, untouched, because a later edit does not erase what came before it. That is precisely what a fresh-repo step exists to handle.
The surprise was a different file entirely.
The file that recorded the removal
When I made that earlier removal, I had done the responsible-looking thing and written a redaction log: a markdown file documenting what I took out and why. The intention was good. The contents were the problem.
The log quoted the removed line verbatim. It named the three products as the reason for the removal. And then, to show its working, it included the search pattern it had used to confirm the cleanup: a list of ten names to grep for. Six of those names appeared nowhere else in the repository. They existed only in the log. The company name I had banned from everything public was in there too, and so was my GitHub handle.
So a three-name removal in one old commit had become a ten-name disclosure at the current tip. The file even carried a cheerful header saying it could be deleted before the repo went public. But deleting it in the latest commit would have left it intact in the five commits behind it, still cloneable by anyone.
Two smaller findings turned up in the same pass, both invisible to a scan of file contents because they live in git’s metadata: my personal email address was the commit author on all seven commits, and one commit subject line spelled out the same three product names.
The detail worth sitting with is this. The two files that actually do the work, the ones anyone would install, were clean the whole time. Every finding was in the scaffolding around the product. The part nobody uses was the part that leaked.
Why I did not just delete the file
The obvious fix is to delete the log and rewrite the commits that carried the names. I did not do that. I rebuilt the repository from the current clean working tree as a single fresh commit and pushed that as the public one. The old repo stayed private as an archive.
Rewriting history is slower and easier to get wrong, and for a two-file pack the history was close to worthless anyway: the changelog tells the version story better than the commits ever did. A tip-only fix would have left every finding sitting in the commits behind it, and would have forced me to reopen a naming decision I had deliberately locked months earlier. One fresh commit made all of that moot at once. I also set the commit author to a public address before that first commit, so the metadata started clean rather than being scrubbed later.
The pattern I keep tripping over
This was the fourth time in a few weeks that something I had written to describe a thing turned out to misrepresent it, always in the flattering direction. A breach-response plan that stated a safeguard was on when it was off. A test suite whose passing message implied wider coverage than it had. A short post drafted from a stale note that understated a real gap. And now a redaction log that named more than was ever redacted.
The common thread: a document written to demonstrate diligence quietly becomes a second source of truth, and nobody audits it as hard as they audit the thing it describes. The redaction log is the sharpest case, because its subject matter is the sensitive material itself. A record of what you removed, if it names what you removed, is just the disclosure with an explanation stapled to it.
What I changed
Two rules came out of it. First, a record of a redaction never quotes the value it redacted. It describes the class of thing removed and the reason, and stops there. “Three unreleased product names, removed for confidentiality” tells a future reader everything useful and leaks nothing. Second, anything found in history is not fixed by a new commit. If a sensitive value is in any past commit, I treat the repository as compromised and rebuild it clean, rather than patching the tip and hoping nobody clones the rest.
The uncomfortable takeaway
My scan was green, my positive control proved the scanner worked, and I still came within one manual sweep of publishing a list of names that existed nowhere except in the file documenting how carefully I had removed them. A clean result from a good tool only ever tells you about the thing that tool looks for. The most useful question before you make anything public is not “did the scan pass,” it is “what does this tool not look at,” and then going to look there yourself.
Compliant by Design is a fortnightly series on building AI products under Australian regulation — Privacy Act, APPs, NDB, health data — written from inside the work, not above it.