← Adam Garceau

I asked 1,000 fake customers before I built anything

How close synthetic audiences are to real ones, and where they lie.

Here's a move that sounds fake and works anyway. Before I build a product, write an ad, or pick a name, I run it past about a thousand simulated customers first.

Not random ones. I build them out of real research. You feed an AI your actual customer call recordings, or if you don't have those, you point it at Reddit, reviews, and forums where your customers already talk. It pulls out the real segments, the real objections, the words people actually use. Then it answers as those people. You get scores, objections, and a read on what would land, in a few minutes, for basically free.

The obvious question is: are the fake answers any good? So I went and read the research.

The best study on this comes out of Stanford and Google DeepMind (Park et al., 2024). When they built these agents from real interview data, the synthetic answers matched a person's own answers about 85 percent as well as that person matched themselves two weeks later. That's close. And here's the part that matters for me: when they used generic made-up personas instead of real data, the accuracy dropped hard. The grounding in real customer language is the whole game. Cheap and lazy gets you garbage. Cheap and grounded gets you a real read.

Now the honest part, because I'd rather tell you the ceiling than sell you a miracle. Synthetic audiences have a known flaw: they're a little too nice. They'll praise things that real people go on to reject. So I treat them as a fast, cheap filter, not a verdict. They kill the obvious losers and surface the objections before I spend real money or real customer goodwill. Then real people, spending real dollars, are the only thing that settles it.

That's the whole method. Get to ninety percent of the answer for free and in minutes, then let the real world take it the rest of the way. It used to take an ad budget to learn what a grounded synthetic read now tells you first.

Sources: Park et al. 2024 (Stanford/DeepMind), Bisbee et al. 2024 (the honest rebuttal), Nielsen Norman Group on sycophancy. Fuller writeup: the evidence, with receipts.

I'm a marketing director who builds his own tools. More at adamgarceau.com.