The thing my agent pulled up was an eval framework inside my own setup. Workflows for running prompt evals. A suite library. A judge framework. Tools for capturing transcripts and turning fails into new work items. It is real. It is full. It has been sitting on my disk for weeks. I had not touched it.
I looked at it and felt like an impostor. Same feeling that hits anybody who inherits a codebase. The "did I really build this" voice. The "I do not understand every line, so do I really own this" voice. The voice that says you are running an AI shop while you cannot recite the inner parts of every tool in your own setup.
Then I remembered. That is how I spent the last decade doing offensive security.
What pentesting actually teaches you
When I engage a target as a pen tester, I do not understand every line of code in the system I am breaking into. I do not understand the JVM inner parts. I do not understand the OS kernel under the app. I do not understand the inner workings of Apache. Or Tomcat. Or the Spring framework. Or the database driver. Or the load balancer. Or the firewall. Or the WAF.
What I understand is enough to know where the seam is.
I know Spring parses XML in a way that can let bad actors slip a ProcessBuilder in. I know SQL Server can pivot into xp_cmdshell. I know a JWT with HS256 and a weak secret can be guessed and re-signed. I know a wrong-set S3 bucket will sometimes hand you a whole client DB. Just add ?listing=1. I know hundreds of these. I know where to look for more.
I do not know how to write Spring from scratch. I do not know how to build a SQL Server query optimizer. I have never built a JWT library. I have never built S3. The inner parts are a wall I have never tried to climb. I have only needed enough of them to find the seam.
The asset is not implementation mastery. The asset is knowing where to look.
What that means for my own AI setup
When I look at the eval framework on my disk, I do not need to understand every helper inside the run tool. I do not need to understand the judge framework's prompt template. I do not need to know how transcripts get parsed.
I need to know enough to say: "this is the place that checks whether my agents are still doing their job. The input is a use case file. The output is a pass-fail. If a test fails, the framework tells me which agent and which case. From there I figure out what to do."
That is the architect's read. It is the same shape as the pen tester's read. Find the seam. Know what the seam is for. Trust that the rest of the system is doing its job. Engage at the seam. Not at the build.
The trap I almost fell into
The impostor trap is a binary. Either you understand every line, or you are a fraud. That binary is real for some kinds of work. Surgeons cannot offload the cutting. They have to do the thing. So do airline pilots. So do nuclear plant operators.
That is not the shape of my work. Or yours, if you run a small business. The shape of our work is judgment. Architecture. Choice. The build is something we can offload. And more and more, something we should.
I have a digital worker on this same machine. An AI agent. She is happy to read every line of the framework and explain it to me. Build me a custom eval against my actual agents. Run it weekly on a cron. Fix things when they break. Only ping me when an actual choice has to be made. The build is her job. The shape is mine.
The framework I felt like I did not own was always mine to drive. I just had to stop treating "drive" as the same thing as "understand every line of code."
This is the operator stance for AI
I write this for the small business owner who reads the AI news and feels buried. You can not keep up. You do not have to. You need to know what the seams are. You need to know which tools to reach for. And when. You need to know what a working AI agent looks like. And what a broken one looks like. You need to know enough to engage.
The agent does the build. The agent reads the docs you do not have time to read. The agent runs the eval framework you do not have time to run. The agent watches the codebase you inherited and tells you what is going on inside.
Your job is to know where the seams are.
Mine is to make sure that when you show up to engage, the seams are clearly marked.
I do not have to understand all of it. I have to understand enough.