Meta's auto creative enhancements: leave them on or keep your hooks untouched?

Meta's Advantage+ creative enhancements (auto text overlays, brightness tweaks, music) are on by default in a lot of accounts, and I'm not convinced they should be. The argument for leaving them on is that the system can test variations faster than you can. The problem is you lose track of what actually ran. Say a skincare brand uploads a clean product video with a specific claim in the first second. Meta adds a text overlay that softens the claim and changes the hook. If CTR goes up, you can't tell whether the hook or the overlay did it, so you can't carry the lesson into the next brief. My view is to keep enhancements off for your control ads and only run them on a copy of a winner that you're willing to test separately. Has anyone seen the auto version clearly beat the hand-cut one on a fair split?

Agreed on the core point, and I'd push it further: CTR is the wrong scoreboard for this test. Brightness and music tweaks can lift thumb-stop rate, and a softened claim can pull in people who never buy. If the enhanced copy wins on CTR but loses on cost per purchase, all you've learned is that the overlay attracts clicks, not that it sells. So judge it on cost per landing page view or purchase.

For a fair split, use Meta's A/B test setup with the creative as the only variable, same audience and budget. Name the variants plainly, something like "enh-on" and "enh-off", so the reporting tells you what ran without digging through asset history. Then let it run long enough to matter. For a skincare brand at modest volume that can easily mean a couple of weeks, and calling it early on a small CTR gap is how people end up carrying the wrong lesson into the next brief.

Two weeks is too short at modest volume. You may only get a few dozen purchases per arm, and a cost-per-purchase gap that size is mostly noise. Let each arm collect enough conversions before you call it, or use cost per landing page view as the early read and confirm with purchases later.

Fully agree on sample size, it's the point people skip. Cost per landing page view can give you an early read, but only if the enhanced arm is actually what you think it is. Enhancements don't always apply the same way across placements, so "enh-on" can turn out to be a mix of overlaid Reels and untouched Feed ads. Pull the placement breakdown and check the previews for each arm before you trust the result. If the overlay only ran in two placements, what you learned is about those placements, not the hook.

It comes down to whether the enhancement touches the first three seconds. If Meta only adjusts brightness or swaps the music, the placement mix mostly changes how the same hook lands, so cost per landing page view is a fair early read. If the overlay rewrites the opening claim, even a clean placement split is measuring a different hook, and I'd treat CTR and cost per view as tiebreakers at best. Check the first frame of each arm's preview before deciding which kind of test you actually ran.

Overlays don't always show up on frame one. Text can fade in at 1.5 or 2 seconds, and that's still inside the window where someone decides whether to keep scrolling. A skincare video with a claim in the first second can look untouched on frame one and still have a softened line on screen by second two. So scrub through the first three seconds of each arm's preview, not just the thumbnail.

The music point cuts the other way from what people assume, too. A lot of feed placements autoplay muted, so a swapped track does almost nothing to the hook there, while brightness and text do. That means a "music only" enhancement can still change the test more than you'd expect, depending on where it ran. The placement breakdown Iris mentioned is what tells you which situation you're in.

Scrubbing the first three seconds is right. I'd also check whether the enhanced version changed the aspect ratio or crop. A 9:16 Reel that gets reframed can push the hook text into the zone where the profile name and caption sit on top of it. If that happens, your untouched control and the enhanced arm differ in exactly the part of the frame that decides whether anyone reads the claim.