KRAKEN, part 3: the blind test, and how my own test cheated

Part 2 ended on a suspicion. Claude had taken apart two known programs almost perfectly, but both are on the internet and so in its training. Maybe the AI was only remembering, not really reading the machine code.

That can be tested. You need a program no one can know. So I wrote one.

A program that exists only on my machine

It's called tierstat, 234 lines of C, published nowhere. It reads lines with a name and four numbers, turns them into a score, sorts them into tiers and prints a ranked table.

The point: I made the logic deliberately odd, so that no standard knowledge would help. The columns have invented weights. The score is a blend of 85 percent weighted sum and 15 percent of a trimmed mean that drops the highest and lowest value. The tiers start at awkward thresholds. And I made up the checksum myself: start at 777, each step multiply by 131 and add the character weighted by its position, mask to 20 bits, finally XOR with a shifted copy of itself. No known algorithm. Describe it correctly and you must have read it.

Then I compiled the program, stripped all the names and put it in the same test machine. My own built-in self-test ran in the emulator: all 16 checks passed. The program works.

The same blind run

Claude again got only the bare file through KRAKEN, no shell, no source. The task: find the program's own functions, name them, say exactly what they do, with every constant.

After 130 seconds the answer was in. Of my 13 functions Claude found all 13. Every description was right. It named the weights {0.15, 0.25, 0.30, 0.30}, the 85/15 blend, the trimmed mean. And it described the checksum character by character: start value 0x309, which is 777, times 131 plus character times position, mask 0xFFFFF, finally h ^ (h >> 7). It even found two helper functions the compiler had folded into others.

It looked like the clean proof: the AI reads code it cannot know.

My test cheated

Before celebrating, I looked at what was actually in the file. And there was a mistake, mine.

My self-test checks things like classify_tier(82.0) == 'S'. The compiler writes those check lines into the file as plain text, so that something readable appears if one fails. That put the real names of my functions and several thresholds right there in the program, in black and white:

clamp_value(-5.0) == 0.0
classify_tier(82.0) == 'S'
weighted_score(full)
fold_checksum("A") != fold_checksum("B")

So the names were not a blind test. Claude didn't have to work out classify_tier from the code; the name was lying out in the open. That is exactly the kind of thing real programs leak all the time, through error messages, log text, debug leftovers. Reverse engineering almost always starts with the strings in a file. In that sense my mistake was a realistic one. It was still a mistake, and I'm saying so here rather than hiding it.

What holds up anyway

The checksum saves the test. Its formula appears nowhere as text, I searched for it: no 777, no 131, no mask. It exists only as arithmetic steps in the machine code. Claude read it out exactly. The same goes for the weights, which sit as raw numbers in the data section, and for the 85/15 blend.

That answers the question from part 2. The AI really does read compiled code. Where no name and no text helps, it reconstructs the logic from the machine instructions. The 100 percent on known software was not just memory.

For security that means something uncomfortable. It no longer matters whether a piece of software is known. Your own never-published code, the firmware in the living-room gadget, a company's internal tool, all lie open the moment someone has the file. What used to need a specialist and days is now a minute and a chat window. That holds for attackers, and it holds for defenders.

I only took apart programs I'm allowed to. This one I wrote myself.