ChatGPT Ads as a test channel: how much budget before you can judge it?

I have been reading about ChatGPT Ads and I keep coming back to one problem. We have years of benchmarks for Google and Meta, but very little for a surface where the user is in the middle of a conversation, not scrolling a feed or typing a search query.

My working hypothesis is that the intent signal may be strong but the path to conversion is harder to see. A click might be the end of a long research chat, so a last-click view could undervalue the channel or credit it for something it only nudged.

So the real debate is about setup, not just performance. Do you start with a small fixed test budget and a written stop rule, such as a set cost per qualified lead or a set number of conversions by a date? Or do you wait until you have a proven baseline on another channel and then compare? Waiting feels safer, but you may never get clean data if you only look at the channel after everyone else has already decided.

I would like to hear how others define a kill threshold for a channel where they do not yet trust the attribution. What did you measure first, and what did you ignore?

Elena, I'd push back on the idea that the bigger risk is undervaluing the channel. The more common failure is the opposite: a new surface gets credit for conversions that were already arriving through branded search or direct traffic, and it looks excellent for a month. Before deciding on the budget, check for that.

Compare conversions from the new channel's landing page with your branded query volume over the same dates. If both rise together, the channel may be relabelling demand you already had, not creating it.

On the kill threshold, a fixed date tells you less than a fixed sample size. At a 3 percent conversion rate, you need roughly 1,000 clicks before a cost per conversion means much more than noise. Size the test budget to reach that, and write the stop rule before launch.

I'd keep the rule on cost per qualified lead, and I'd decide now what to ignore: assisted and view-through numbers stay out of the judgement until the baseline is stable.

Theo, I like the branded-query check, but I'd tighten the sample rule. At a 3 percent conversion rate, 1,000 clicks gives you about 30 conversions. The 95 percent range around 30 runs roughly a third either way, so a cost per lead that looks 40 percent better than target could still be noise. A strict stop rule should be based on the number of conversions reached, not only the click count.

I'd also treat the first test as a creative test before it is a channel test. Say you run a dental clinic and your ad reads like a generic "book a consultation" line. Someone asking an assistant about clear aligners is in a different frame from someone scrolling a feed, and the message has to answer the question they are already asking. If you judge the surface on one weak message, you may kill a channel that would have worked with a better hook. Split two or three angles inside the same budget, and read the channel verdict only after you know which message the traffic responds to.

Mira, I agree the 30-conversion point matters, but for a local service business the sample problem is harder than the maths suggests. A clinic running a small test may need months to collect 30 qualified leads, and by then the season has moved on. I would pair the conversion target with a hard spend cap, so the test ends on whichever comes first, and accept that the verdict is directional.

Theo, for local businesses I would adapt the branded-query check. The closest equivalent is often Google Business Profile activity: calls, direction requests and website clicks from the profile. If those rise together with paid-channel calls, the new channel may be relabelling demand you already had.

One practical step before launch: give the new channel its own call-tracking number that is not used anywhere else, and log how many calls you cannot attribute. If that unattributed share is large, your cost per lead is closer to a guess than a measurement, and the stop rule should say so in writing.

Marco, I'd go a step further on the unattributed calls. Don't just count them, ask. A single question at the start of each call, like "where did you first hear about us?", logged against the call, gives you a rough split between paid, profile and word of mouth. It's imperfect, but it turns a blank into a directional number you can argue about.

I'd also write the definition of "qualified" into the stop rule before launch and freeze it. If the sales team tightens what counts as qualified halfway through, cost per qualified lead moves for reasons that have nothing to do with the channel. Note who is allowed to change the definition, and when.

Finally, I'd make the rule asymmetric. It should be easy to kill the channel when the hard cap or the stop condition is hit, and harder to scale it after one good stretch. Promoting the channel should need a second window of data showing the same result, not one lucky month.