Meta creative testing: one ad set per concept or many creatives in one ad set?

Still going back and forth on this one. For creative testing on Meta, do you give each new concept its own ad set so it gets a clean budget and a fair read, or do you put five or six creatives into one ad set and let delivery sort them out? Take a skincare brand at around 40 dollars a day. Separate ad sets give cleaner data, but each one gets so little budget that most never leave learning, and the winner you pick is often just noise. One ad set is faster because the algorithm picks a favorite quickly, but an early loser can get starved before it's had a real shot. My lean is one ad set per testing batch on the same audience, then moving the winners into a separate scaling ad set. Curious how others handle budgets this small.

One ad set is the right call, but I'd change what it optimizes for. At $40 a day, a purchase-optimized ad set stays stuck in learning. Meta wants roughly 50 optimization events a week per ad set, which works out to about 7 purchases a day. If your cost per purchase is $30, $40 buys you one or two, so the algorithm never settles.

For the test ad set, optimize for landing page views or add to cart instead. That gives you enough events to actually read the creatives. Judge the concepts on hook rate and CTR, and only move the winners into a purchase-optimized scaling ad set once they can carry real volume.

Hitting 50 events only means the ad set has left learning. It says nothing about how precise your creative comparison is. At $40 a day the events get split unevenly across creatives, so each one gets a few dozen a week at best.

That's enough to spot a big gap, not a small one. For budgets this size my rule of thumb is to make the concepts in a batch clearly different: a different angle, a different person on screen, or a different offer, not five versions of the same hook. If the difference you're hoping to find is small, you'd need several times more spend to see it, and at $40 a day you won't get there.

I'd also judge the top two or three on CTR, then confirm on purchases once they move into the scaling ad set, because a curiosity hook can win clicks and still lose buyers.

Quick math on the "few dozen" figure: $40 a day is about $280 a week, so six creatives get roughly $47 each. At a $1 CPC that's around 45 clicks per creative per week, which means CTR gaps smaller than about 30% are mostly noise at this spend.

I'd set that threshold higher. With 45 clicks the standard error is around 15%, and comparing two creatives roughly doubles the noise, so you need a gap closer to 40% before it starts to look like a real difference. That also assumes Meta splits spend evenly, which it rarely does. The creative it favors early gets more impressions, and the weaker one can end up with 20 clicks, which makes the comparison shakier still. At this spend I'd treat the CTR ranking as a shortlist, not a verdict.

Your 40% figure holds up if you want a two-sided 95% comparison. The difference between two counts of 45 clicks has about 21% standard error, so 40% is roughly two sigma. Kai's 30% would be closer to a coin flip.

For the shortlist, I'd also look at a metric with more events behind it. On video creatives, 3-second plays usually run several times the click count at the same spend, so hook rate gives a tighter read than CTR. Check it alongside CTR, as long as you're comparing the same creative format.