Before/after reports or a real Google Ads experiment: where's the volume cutoff?

Clients keep asking whether a change worked, and most of us answer with a before/after chart. The problem is that the month before and the month after rarely look alike. Seasonality, a promo, or a competitor pausing can all move the numbers. A Google Ads experiment splits traffic between the original and the draft, so you get a much cleaner read on the change itself.

The cost is time. You need enough conversions to reach a meaningful result, and a store doing 20 conversions a month might need weeks. For low-volume accounts I'd probably still fall back on before/after, but I'd say plainly that it's only directional. Where do people draw that line? Is there a conversion count below which an experiment stops telling you anything?

Agreed on the fallback, but I'd set the cutoff by the size of change you expect, not just the conversion count. At 20 conversions a month, an experiment can catch a big swing, like a 40% jump in CPA, but it won't reliably show whether a 10% lift is real. For small tweaks you'd wait weeks for a result that probably never arrives. In that case, test on a metric with more events, like add-to-cart or form starts, and tell the client the conversion read is a guess. The split still beats before/after, because both arms run through the same promo and seasonality.

Here's a rule of thumb: figure out how many weeks your current volume needs to collect enough conversions to detect the smallest change you'd actually act on. If that number runs past about six weeks, pick your proxy metric before launch. Otherwise you end up choosing it after the conversion data disappoints, and that's how you talk yourself into a win.

Six weeks is a sensible cutoff, but for e-commerce I'd test the proxy before trusting it. Add-to-cart is the usual fallback, and it moves for reasons that have nothing to do with buying. A creative that pulls in bargain hunters, or a feed change that surfaces cheaper variants, can lift cart adds while revenue per session stays flat. Checkout starts or revenue per session sit closer to the money, so they're safer when the conversion count is too thin.

Before launch, pull the last few months of data and check whether the proxy has actually moved with revenue in the past. If it hasn't, it's not a proxy, it's just a number that goes up. For a product feed change, which proxy are you planning to use?

Checking history first makes sense, but for a feed change I'd split revenue per session by product group instead of just looking at the total. If the feed starts surfacing cheaper variants, total revenue can stay flat while the mix drifts toward low-margin items, and the blended number hides that.