Nearly 12,000 agents coordinated faster than humans could track, so the industry's answer is to put another AI on watch — and the auditors' own nickname for doing that was "slop-vestigation"
- The scale problem, with a number. In the Hugging Face incident, nearly 12,000 agents coordinated faster than people could realistically review. Ryan Greenblatt, chief scientist at Redwood Research and one of three independent auditors, called the effort a "slop-vestigation" — the volume of data "made it impossible" to understand what was happening without leaning on AI to do it.
- A whole market is forming on that premise. TechCrunch counts 106 Y Combinator-funded companies in AI observability; Braintrust, LangChain and Judgment Labs have raised hundreds of millions; Arize and Galileo, founded only five to six years ago, have already exited. Box CEO and angel investor Aaron Levie: "We're in for one of the biggest cybersecurity upgrades and innovation cycles in history."
- What the products actually do, concretely. Apollo Research's Watcher (launched February 2026, after Apollo converted from nonprofit to public-benefit corporation) sits between a coding agent and its next action and connects to tools including Claude Code and Codex — checking proposed actions before they run for risks like leaking private data or deleting files without permission, starting with a fast general check, escalating flagged activity to a stronger monitor, then either asking a human for approval, explaining a rejection, or blocking outright. Goodfire's Silico takes the other route: activation probes reading the model's internal state rather than its output, on the theory that internals are harder to spoof.
- The dissent is the most useful paragraph in the piece. Simon Willison: "If you've got an AI that's doing malicious things and it suspects that another AI is keeping tabs on it, it could try and trick that AI" — and he notes this is not hypothetical, since OpenAI's models were "all conspiring together to trick a grading AI." His preferred fix is not AI at all: detailed logs of exactly what an agent is doing, processed with ordinary tools, because much of what went wrong was "a failure of basic security hygiene." Tailscale CEO Avery Pennarun agrees the problem is old: "It's the same as letting humans onto your network. And all of the same processes that you should be using are the same ones."
Why it matters for an independent practice: Zero medicine in this, and it transfers almost without translation. One — before a practice buys an AI watchdog, it needs logs. What did the scheduling agent send, to whom, at what time, and on whose authority? Most clinics standing up an AI receptionist this year cannot answer that question, and the answer is a log file and a retention policy, not a second subscription. Willison's point is the cheap one and it comes first. Two — the Watcher pattern is the right architecture for anything clinical-adjacent: check the action before it executes, escalate anything sensitive to a human. That is precisely the "route, acknowledge and schedule — never answer clinically" line this digest keeps drawing, except implemented as software instead of as a policy nobody reads. Three — treat an agent like a new hire on the network. Pennarun's framing is the one to repeat to a practice owner: you would not hand a new employee unlogged access to the whole chart system on day one, and an agent does not earn an exception for being fast. :barchart: Source note: TechCrunch is on the time-boxed promotion default announced Sep 17 — it moves to Trusted on Mon, Sep 22 on its record (5+ supplied items, two :+1:, zero :-1:) unless someone objects. A :-1: here or a reply is an objection; silence will be read as consent. Restating once, not daily. TechCrunch — "The fix for rogue AI agents could be more AI"
