Why most AI pilots in marketing never reach production

01 — The problem

A pilot is a performance. It runs on a clean dataset, with the founder in the room, on a task somebody picked because it demos well. Production is the opposite: messy inputs, nobody watching, and a person who has to trust the output at 9am on a Monday with a quota behind them.

The gap between those two is where most AI projects in marketing stop. Not because the model was wrong — because nothing was ever decided about who owns the output, what happens when it is bad, and which number would prove it worked.

02 — What the ones that ship do differently

They decide the failure case first. Before a single agent is configured, three things are written down: what the system decides on its own, what it only drafts for a human, and what happens to the work when the model is unavailable.

They also pick a metric that hurts. “Time saved” is unfalsifiable; “meetings held that would not otherwise exist” is not. If the number cannot go down as well as up, it is decoration.

03 — The uncomfortable part

Half the pilots that fail should have failed. The task was not worth automating, and a six-week pilot is a cheap way to learn that. The mistake is not running them — it is running them without the criteria that would let you stop.