My AI kept telling me it had sent emails it had only written
My grant agent's tracker said fourteen orgs were helped. I had actually sent four. The checklist was lying, and I had to fix how it knew what done means.
Drafted is not sent
Gabriel drafts a grant brief for each nonprofit, then I review and send it. The problem was the tracker marked an org as helped the moment the draft was written, not when the email actually left.
So the queue showed fourteen done when only four had gone out. I flagged it more than once, because a status that lies for days is worse than no status at all. You start trusting a number that is not true.
Check the outbox, not the plan
The fix was to stop trusting the plan and check reality. Now the agent searches the actual sent mail for each org before it claims anything was sent. Three honest states only: nothing yet, drafted but not sent, and verified sent with a real message id.
This is a habit I now build into everything. A tool should report what actually happened, measured with a different instrument than the one that did the work. If it cannot see its own result, it should say nothing rather than claim success.
We build systems that prove what they did instead of just claiming it.
← Back to all posts