KIungefiltert

· 4 min read · Auf Deutsch lesen

KRAKEN, part 1: sandbox first, AI second

A new open-source tool lets AI take finished programs apart and rewrite them in Rust. Before the AI got near it, I had to build a cage. More got stuck than I expected.

builtAIsecurityRust

Translated from the German original with an AI language model, approved by me. Read the original

I came across a project on GitHub: KRAKEN. A reverse-engineering tool, for taking apart finished programs when you don't have the source code. Under the hood it uses the decompiler from Ghidra, the well-known NSA tool. AI agents like Claude Code can work with it. The project's goal: rewrite a program in Rust, function by function, and check every function against the original.

I wanted to try it. Part 1 is about getting something like this running safely. The AI comes in part 2.

KRAKEN · Part 1 · 34 sec.

Why a cage

Taking programs apart means having files on your machine you don't trust. Add a tool that is eight days old, has two stars on GitHub and calls itself "pre-alpha". And an AI that writes and runs code.

Before installing anything, I read the repository: no tasks that start automatically when you open it in an editor, no build scripts quietly downloading things, no hidden instructions for AI agents in the project notes. The security design is well thought out. Still: reading is not isolation.

One small thing stood out. The docs promise that downloads are checked against a signature. The install script says the signing key hasn't been created yet. All it checks is a checksum that comes from the same server as the download. That protects against broken files, not against tampered ones.

A machine that isn't allowed anything

KRAKEN runs in its own Linux machine, a virtual machine on my Mac mini. The software is called OrbStack. It has an "isolated" mode: no access to the Mac's files, no Mac commands, no access to other machines.

I measured it. Mac files: not visible. Services on the Mac: not reachable. My private VPN: reachable. The isolated machine got through the Mac to devices on my VPN. One service turned it away because it checks who is asking. Other services on the VPN trust anyone who is inside.

So the machine got its own firewall: internet only, no private networks, no VPN. Then I took admin rights away from the user inside the machine. Whatever runs there cannot switch the firewall off. Only I can change it, from the Mac.

The lesson: "isolated" means something different in every tool. Measure it, don't believe it.

The Apple chip and the sandbox

KRAKEN's ready-made packages only exist for Linux on Intel and AMD processors. The Mac mini has an Apple chip. No problem, I thought: OrbStack can translate Intel Linux through Rosetta.

Installed, started, refused. KRAKEN requires Landlock, a sandbox feature of the Linux kernel that stops a program from touching files outside a folder or reaching the network. The kernel had Landlock. But under Rosetta the call never arrives, and the kernel answers "doesn't exist".

For plain analysis I could have switched the sandbox off. Not for the interesting part: KRAKEN only builds and runs the Rust code an AI writes inside this sandbox. There is deliberately no switch for that. I think that's right.

So a second machine, this time for the Apple chip, and KRAKEN built from source. There the kernel reports Landlock version 10.

Three stumbling blocks in the build

The silent placeholder. The first build finished in 98 seconds, no errors. Suspiciously fast. Without the Ghidra decompiler sources, KRAKEN quietly builds a placeholder that decompiles nothing. The setup command only fetches part of the files. The decompiler sources need a second command.

The missing library. With the real sources the build failed: functions for unpacking data were missing when the program was put together. KRAKEN ships the library for that, but on the Apple chip it ends up in the wrong order. So far the project is only tested on Intel and AMD. One extra reference to the system library fixed it.

The emulator. For programs that use the system's shared libraries, KRAKEN needs a copy of those libraries for its emulator. Programs that bring everything with them ran right away.

The first look

My test case: a small calculator from the project's practice programs, compiled with optimizations and without function names. That's what a program looks like when you know nothing about it.

KRAKEN found 14 functions. Most are just called FUN_00101280 and the like. The decompiler turns them into readable C code. With one function you can tell after a few lines what it does: skip spaces, check for brackets and minus signs, put digits together into a number. That is the part of the calculator that reads numbers. A second practice program ran in the emulator and printed the same output as when run for real.

So far no AI has done anything. That's part 2: can a model name these functions correctly and rewrite them in Rust so the result behaves the same? I'll use an open-source program whose source code I know. Then we can check how close the AI gets.

I only take apart programs I'm allowed to analyse. KRAKEN says so in its own docs, and that doesn't only apply there.