Heads up before anything else: I'm not a developer. The test results below are solid and repeatable, but the guess at the bottom about what's causing it might be nonsense. I used Claude Code to help me bisect this, which is how a non-coder ended up with a 47-file test matrix.
Short version. A two-line, 207-byte text file fails the final safety check every single time. A 100-row file containing those same rows plus a lot more sensitive data exports fine.
The file
Lantern Bay Hospitality | FGT60FTK23005512 | 2027-10-20 | lаntern@lanternbay.example | 864-555-0142
Alder & Finch Advisory | FGT60FTK23008801 | 2027-11-30 | reception@alderfinch.example | 864-555-0169
Two data lines, one blank line between them, two spaces at the start of each data line.
The a in lаntern is a Cyrillic а, U+0430, not a normal U+0061. It's the only non-ASCII character in the file. I put it there deliberately as a test case for lookalike characters. The address is 26 characters long but 27 bytes.
All the data is made up. .example domains, phone numbers from the 555-01xx range that's reserved for fiction.
It's not flaky. Two byte-identical copies of the same file (checked with cmp) failed on separate runs, and I saw no randomness anywhere across the 47 files.
What has to be present
Five things. Take any single one away and the file passes. Each row below is a control file that differs from a failing one in exactly that one respect.
| Has to be there |
Control I tested |
Result |
| A non-ASCII character inside something that gets redacted |
same file with a normal a |
passes |
| A blank line somewhere in the file |
same rows, no blank line |
passes |
| A serial or a date column |
both columns deleted |
passes |
| A customer name column |
name column deleted |
passes |
| A phone column |
phone column deleted |
passes |
On the third one, either column on its own is enough. I deleted just the serial, kept the date, still failed. Deleted just the date, kept the serial, still failed.
Things I thought it was and it isn't
Each of these got its own control file:
- How many emails are in the file. It fails with 2 and passes with 3, 4 and 5.
- File size. 100 rows passes, 2 rows fails.
- A second multi-byte character (an em dash). Passes with one present, fails with none.
- Indentation. Fails flush left too.
- Extra non-data lines mixed in with the rows. Fails with none present.
- Where the Cyrillic character sits relative to everything else. Fails either way round.
- A partial value repeated in plain text, like a bare
0147 next to a redacted 864-555-0147. Passes.
- The same value written two ways,
4111-1111-1111-1111 and 4111 1111 1111 1111. Passes.
- The lookalike character in ordinary prose instead of a delimited row. Passes.
One result that doesn't fit
I took a 21-row slice of my source document, which includes the Cyrillic row, and stuck a blank line in right after that row. That file has all five conditions and it passes. It's the only file out of 47 that contradicts the pattern.
I'm reporting it rather than quietly leaving it out. Maybe there's a size threshold where something different kicks in.
The guess, which is only a guess
That 26 characters / 27 bytes mismatch is the thing that keeps catching my eye. If positions get counted one way and the text gets cut and spliced another way, anything after the Cyrillic character would land in the wrong place. If that's what's happening then your checker is doing its job correctly and the real problem is upstream of it.
That doesn't explain why all five conditions have to be true at once, though, so treat it as a hunch from someone who doesn't write code.
Related
The reason this took 47 files is that the error message doesn't identify the content it's complaining about. Filed separately as #3.
I've got all 47 files with their pass and fail results and can attach whatever subset is useful. Happy to test a build.
Heads up before anything else: I'm not a developer. The test results below are solid and repeatable, but the guess at the bottom about what's causing it might be nonsense. I used Claude Code to help me bisect this, which is how a non-coder ended up with a 47-file test matrix.
Short version. A two-line, 207-byte text file fails the final safety check every single time. A 100-row file containing those same rows plus a lot more sensitive data exports fine.
The file
Two data lines, one blank line between them, two spaces at the start of each data line.
The
ainlаnternis a Cyrillic а, U+0430, not a normal U+0061. It's the only non-ASCII character in the file. I put it there deliberately as a test case for lookalike characters. The address is 26 characters long but 27 bytes.All the data is made up.
.exampledomains, phone numbers from the 555-01xx range that's reserved for fiction.It's not flaky. Two byte-identical copies of the same file (checked with
cmp) failed on separate runs, and I saw no randomness anywhere across the 47 files.What has to be present
Five things. Take any single one away and the file passes. Each row below is a control file that differs from a failing one in exactly that one respect.
aOn the third one, either column on its own is enough. I deleted just the serial, kept the date, still failed. Deleted just the date, kept the serial, still failed.
Things I thought it was and it isn't
Each of these got its own control file:
0147next to a redacted864-555-0147. Passes.4111-1111-1111-1111and4111 1111 1111 1111. Passes.One result that doesn't fit
I took a 21-row slice of my source document, which includes the Cyrillic row, and stuck a blank line in right after that row. That file has all five conditions and it passes. It's the only file out of 47 that contradicts the pattern.
I'm reporting it rather than quietly leaving it out. Maybe there's a size threshold where something different kicks in.
The guess, which is only a guess
That 26 characters / 27 bytes mismatch is the thing that keeps catching my eye. If positions get counted one way and the text gets cut and spliced another way, anything after the Cyrillic character would land in the wrong place. If that's what's happening then your checker is doing its job correctly and the real problem is upstream of it.
That doesn't explain why all five conditions have to be true at once, though, so treat it as a hunch from someone who doesn't write code.
Related
The reason this took 47 files is that the error message doesn't identify the content it's complaining about. Filed separately as #3.
I've got all 47 files with their pass and fail results and can attach whatever subset is useful. Happy to test a build.