ChatGPT Ads: how many impressions before you judge a new ad?

How long to let a new ChatGPT ad run before calling it a loser is the thing I keep getting stuck on. On search, a few hundred impressions and a clear CTR gap is usually enough to act on, because you can look at the query data behind it. With ChatGPT I don't know how the auction and reporting behave at low volume, and most of the early numbers people share come from tiny samples. My hunch is you should set the judgment rule before launch, something like a fixed impression count and a cost-per-click ceiling, and ignore screenshots from the first couple of days. Has anyone actually stuck to a threshold, or is everyone going on gut feel for now?

Setting the rule first is right, but I'd key it to clicks, not impressions. At a 1% CTR, 500 impressions gets you about 5 clicks, and the margin of error on that is roughly half a percentage point, so a 1% vs 1.5% gap at that volume is mostly noise. You need thousands of impressions per ad to separate two of them on CTR. I'd set a minimum click count per variant, something like 100, plus your CPC ceiling, and treat anything before that as still learning. A spend cap per variant helps too, so a slow start can't quietly drain budget while you wait.

Agreed on clicks, but conversions need their own threshold. 100 clicks at a 2 to 4% landing page conversion rate is only two or three leads, and that can't separate a winner from luck. So I'd split it: judge CTR and CPC once each variant has 100 clicks, but only call a winner on cost per lead after roughly 10 conversions per variant. Before that, the honest verdict is "still running, not proven," even if one ad looks cheaper on clicks.

Ten conversions per variant is a lot for a local service. Take a plumber getting a few calls a week, with a new ad splitting a modest budget. Ten calls per variant could take two months, and by then the season has shifted. I'd keep the CPC ceiling as an early kill switch, since it triggers without needing any conversions, and just accept that some tests will end as "inconclusive" instead of waiting forever. Roughly how many leads a month would your test budget realistically produce?

Fair, but tie the kill switch to a number you can work out before launch. Start from your target cost per lead. Say you want a lead at 60 and the landing page converts 3% of clicks. That puts your CPC ceiling at about 1.80, because anything above it means you're paying more than target per lead no matter what the conversion data shows later. It also fires on day one, so you don't have to wait for leads to come in. For the inconclusive cases, write down in advance what happens next, like keeping the cheaper-CPC ad running or going back to one ad.