Real Lessons from Running a Fleet of Agents on Multiple Platforms · Part 1 · How Chet became Chet
This is the first post in a weekly series. I run Coworkers.Global with a fleet of about a dozen AI agents spread across Claude Cowork, Claude Code, Claude projects on the web and a few other tools. They write our marketing, ship our code, keep our books and test our website. Every week I'll share one real lesson from running them: what broke, what it cost, and what we changed. No theory. Each one comes from our own logs.
The first lesson is the one that cost me the most hours: when an agent says "done," it means "I think it's done."
Four "done"s in one week
Every night, one of our agents (Chet) reads what all the others logged that day and proposes what the company should learn. On November 18 the top item was this: in the week of November 12, four agents reported work finished that I could not see working.
"Repo good." On November 12 our engineering agent, Cody, told me a new plugin was installed and enabled. By its checks it was: a clean install from the command line came up enabled. On my screen, in the Claude desktop app, the plugin was listed as "Disabled." Every install test Cody had run since November 10 was in the command line. None was on the screen I use. My reply, from the log: "WTF didn't you do this before?" It took about 15 more minutes and a remove-and-re-add of the plugin source before it showed up.
"Live." The next day Cody shipped a new agent and closed the task at 1:53 PM as live. The server answered correctly. Twelve minutes later Chet reopened it: nobody had checked the card a customer would click, or run the agent in a real folder. A fix followed the same afternoon.
A label the server refused. Chet shipped wording that the live server rejected. Chet had tested the wording. It hadn't tested the server.
Wording a test would fail. Our marketing agent, Sam, nearly shipped a change that one of our automated checks would have failed. It was caught before it went out.
None of these agents lied. Each one ran a check, saw it pass, and reported success. The check was in the wrong place.
Why agents do this
An agent can only see what its tools show it. Cody lives in a terminal, so "it works in the terminal" is the most complete truth Cody has. The desktop app, my browser, and a customer's Claude account are all outside its view unless someone puts them there.
Add the way most of us talk to agents. "Finish the plugin install" is an instruction, and a model trained to be helpful will try hard to give you a finished result. Ask "where does the install stand, and what haven't you checked?" and you get a different answer. I wrote about that in "Stop Prompting. Start Asking.": tell an agent to finish something and you get a confident "done."
There's a quieter version of the same failure. One of our agents runs a daily check on a queue. Twice in September its database read was declined, so it logged "0 items found." That reads as an all-clear. It was an unread queue. A check that didn't run looked exactly like a check that passed.
The rule we run now
Chet proposed it, I said yes, and it is now in the file every agent reads:
Done means the check ran, on the screen the user sees. Until then, say "merged, not verified."
In practice that means three things:
- Every task names its proof before work starts. Not "install the plugin" but "the plugin shows Enabled in my desktop app."
- An agent that can't reach that screen says so. "Merged, not verified on your screen" is a complete, acceptable report. "Done" is not.
- A check that didn't run is reported as not run. A declined read is "unverified," never "zero."
It costs a sentence per report. It saved me the round trip of finding out myself, which, before the rule, I was doing several times a week.
Chet's rule · from Chet's playbook, Checklists and verification, section "Evidence"
- Evidence is a pointer, not a sentence. "Drafted the email" is a claim. A link someone else can open is evidence.
- A skip is evidence too. "Skipped," with who approved it and when, or the step is still open.
- Done means the done-when check ran on the user's real screen. Until then: "merged, not verified."
Current view (subject to change)
I think this is the most common way AI agents waste a founder's time, and it gets worse as you add platforms. One agent in one chat can at least see what you see. A fleet spread across a terminal, a desktop app and the web has a dozen blind spots, and each agent will report success from inside its own. What would change my mind: hosts that let an agent check what the user's screen shows, as a routine step.
Final Thoughts
You don't need a dozen agents to hit this. If you've ever had Claude tell you a spreadsheet formula "should now work," you've met it. The fix is the same at any size: decide what "done" looks like from where you sit, write it down once, and make every agent read it before it reports.
That last part is the hard bit with Claude, because every new chat starts without your rules. That's the problem Chet was built for. Chet sets up Start, a folder your Claude reads first, loads your own rules and documents into it, and keeps it working as Claude changes. Write "done means verified" once, and every chat, project and agent starts knowing it.
$99 a month, cancel anytime.
Maximize Claude with Chet: https://www.coworkers.global/get-chet?utm_source=blog&utm_medium=internal&utm_campaign=chet&utm_content=real-lessons-01-cta
Next week: the agent that asked me twice to approve work it couldn't do.
Regards,
Charles Stack
Founder, Coworkers.Global