Google Ads experiments on a small account: 50/50 split or wait for more data?

Take a campaign that only gets about 20 conversions a month. A 50/50 experiment leaves each arm with roughly 5 conversions over the test window, so even a big swing in cost per conversion can easily be noise. Calling a winner after two weeks mostly means you're reading luck. An 80/20 split doesn't fix it either, since the 20 side never collects enough data to show anything. My view is that at that volume you either run the test much longer or treat it as a directional read, and make the call based on the cost per conversion trend plus which search terms each arm is pulling in. Has anyone actually gotten a clean answer out of a split that small?

Agreed on the math, and it's worse than it looks. With roughly 5 conversions per arm, even a 2x gap in cost per conversion usually won't reach significance, so the test can't really fail. At this volume you're better off testing something that throws off more events, like CTR on ad copy or clicks to the landing page, and using conversions only as a sanity check on the winner.

CTR works fine as a proxy for ad copy, but be careful with it on e-commerce. A headline that pulls more curious clicks can lift CTR while sending people who bounce off the product page, so you end up paying more per session for the same sales. Clicks to the landing page have the same problem, just with cheaper traffic.

A rule of thumb: before the test starts, write down how many conversions per arm you'd need to call it, using a rough calculator, then compare that with what the account can actually produce in your window. If the gap is big, pick a metric that's 3 to 5 times more frequent, like add-to-cart or checkout starts for an online store. Agree in advance that the winner has to hold up on revenue or margin before anyone rolls it out. That way the proxy can't quietly become the goal, and you also have an answer for a client who asks why you didn't just wait for purchases.

Bidding setup can quietly undo this. If either arm uses Smart Bidding with a target tied to add-to-cart, the algorithm is already optimizing for the proxy, so the test partly measures how well the bidder chases carts instead of how the copy performs. A bidder can learn to rack up cheap carts, and that arm looks great until you check revenue. Keep both arms on the same bidding goal and let only the copy differ. Then the proxy is measuring the creative, not the bidder's shortcuts.

I don't think that last part holds. Keeping the same bidding goal doesn't make the copy the only variable. Smart Bidding predicts conversion rate per auction using signals that include the ad itself, so a headline that shifts predicted CTR also changes how hard the bidder pushes and which auctions the ad wins. The arm with more impressions isn't just better copy, it's the copy plus the bidder reacting to it. For a copy test on a small account, I'd run both arms on manual CPC or the same fixed bids for the test window, then switch back to Smart Bidding for the rollout.

Manual CPC doesn't fully fix it either. Ad Rank still factors in expected CTR and ad relevance, so a headline Google expects to get more clicks can win more auctions at the same bid. The copy still steers delivery, you just lose the bidder layer on top.

For a small account, I'd put both ads in the same ad group, keep bids identical, and set ad rotation to "do not optimize" if your campaign type still offers it. Then look at the impression split after a few days. If one ad is getting around 65% or more of impressions, the result is mostly about who got shown, so I wouldn't read the CTR or conversion gap as a copy win yet. You'd need to even out delivery or treat the result as much weaker evidence.

Depends on the ad format, and that's worth checking before you read anything into the impression split. If both ads are fixed copy, "do not optimize" gives you a fairly even rotation, so Elena's 65% line works as a useful tripwire. With Responsive Search Ads the setting is much weaker, because Google mixes headlines and descriptions into different combinations and keeps learning which ones to show. That means an "ad A vs ad B" split can skew inside the ad itself. For RSAs I'd test one headline at a time with pinning, or run separate ads.

Pinning helps, but pinned headlines still limit the combinations Google can serve, so the test isn't clean either. On a small account I'd skip RSA copy tests altogether and run separate ads instead. Which format are you actually testing right now?

Pinning does cut down the combinations, agreed. But I wouldn't lean on the Best/Good/Low labels Google puts on RSA assets. They only rank assets against the other assets in the same ad, and there's no significance check behind them, so a "Best" headline might just be the one that got the early impressions. I'd treat them as a hint for which separate ad to build next, not as a result. Is the goal here more clicks or actual sales?