Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
ROAS BIBLE · Part 1 — Perspective: Meta ads as an answer that works in any industry
Bar height is proportional to the number of purchases (values from the “Results by attribution setting” column)
Don’t look at ROAS. Look at the purchases that keep coming in over time
1
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Key points of this chapter
1
The first test of a good ad is not ROAS (return on ad spend). It is the absolute number: cost per result. The second test is extra metrics such as the attribution setting.
2
Split results into 1-day, 7-day and 28-day click, and some ads get cheaper per purchase as time passes. Keep those ads running, even when their frequency is high.
3
Watch an ad up to 2× BEP (break-even point). Cut tightly at 1.3× and you miss the ads that would have come back later.
4
Asking about the algorithm, detailed targeting, daily budget or how much to scale up is all a system view. The question to ask is whether people respond.
5
A winning creative dies because the pool of people who respond to that appeal is used up. Change the appeal, not the setup.
2
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
1. Case 3: Is the higher-ROAS ad the better ad?
The third case shows results, cost per result and purchase ROAS. On ROAS alone, the ad with the highest ROAS is the best ad. But Meta ads are about adding options and optimizing. Chase ROAS alone and you may run out of options later.
On the course screen, the upper red box is an ad with 5 purchases at ROAS 2.91, and the lower one is an ad with 89 purchases at ROAS 2.78.
3
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Look at the two ads in red boxes. One has 5 purchases at ₩34,022 (≈ $25.6) per purchase (Korean market) and ROAS 2.91 (291%). The other has 89 purchases at ₩47,281 (≈ $35.5) per purchase and ROAS 2.78 (278%). Five purchases against 89 is a big gap in sample size, so the comparison is shaky, and you could say they can’t be compared at all. Still, this table makes two points.
First: the absolute number
Look first at the cost per result. The ad with the lowest cost per result is the first good ad. But an ad like this can burn out fast, so it’s risky unless you build up options.
Second: extra metrics
Look at metrics such as the attribution setting alongside it. You can check whether a conversion came 1 day, 7 days or 28 days after the ad click. To add the attribution setting as a custom metric, search “attribution setting” in the data dashboard of Meta Ads Manager.
4
Supporting diagram · Ranking by ROAS
Supporting diagram
Ranked by ROAS, the #1 ad’s cost per purchase is over 2× BEP
The 10 ads from the Case 3 screen, in the screen’s order. A·B = the two ads the course compares. ROAS has the purchase amount mixed in. The #1 ad (8.80) only scored high because its 3 purchases averaged about ₩1.07M, which is why the course says to look at the absolute number (cost per result) first.
Source: Meta Business Help Center, Website purchase ROAS (return on ad spend) · Cost per result (checked 2026-11-20)
5
Supporting material · 25 ad experiments
Supporting material
Of 25 large ad experiments, only 3 could tell a big win from break-even
Swing in sales per personStd. deviation ≈ 10× the mean
ROI std. error, retail testsMedian 26.1%, CI over 100 pts
To detect a 10-pt ROI gapMedian test must be 62× larger
Course5 purchases (ROAS 2.91) vs 89 (ROAS 2.78): the gap in sample size makes the comparison itself shaky
25 randomized controlled experiments by US retail and finance advertisers. One cell = one experiment; blue = enough power to answer that question reliably (expected t-value above 3). Per-person spend swings about 10× the mean, so error stays large even with 500K+ people per test. The paper’s ROI is profit-based and from randomized tests, so it is defined differently from dashboard ROAS.
Source: Lewis, R. A. and Rao, J. M. (2015), The Unfavorable Economics of Measuring the Returns to Advertising, Quarterly Journal of Economics 130(4), Table III (checked 2026-11-20)
6
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
2. Why we scaled the 278% ROAS ad, not the 291% one
Of the two ads, we raised the budget on the 2.78 ad, not the 2.91 one. The reason is in the “Results by attribution setting” column.
The 2.78 ad has only 69 direct conversions within 1 day of the click. But keep it running and 9 more come in within a week, for 78. Over 28 days, 11 more come in, for 80. As time passes, the cost per purchase keeps falling and overall ROAS improves.
1-day click69direct conversions
▶
7-day click789 more
▶
28-day click8011 more
7
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
In the 1-day, 7-day and 28-day click columns, the ROAS 2.78 ad grows from 69 to 78 to 80, while the ROAS 2.91 ad stops at 4, 5, 5.
Ad
ROAS 2.91 (291%)
ROAS 2.78 (278%)
Cost per purchase
₩34,022
₩47,281
1-day click
4
69
7-day click
5
78 (9 more)
28-day click
5
80 (11 more)
Decision
Budget not raised
Budget raised
People say “once an ad is live, give it 7 days.” Once you factor in the attribution setting, that can be partly true.
8
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
But “one right answer” rules like “the daily budget must be ₩30K (≈ $23)” or “always run an ad for at least 7 days” are the wrong approach. Waiting 7 days is only right when you look at it from the attribution-setting angle.
BEP for this product = ₩60K (≈ $45)
Turn off every ad costing ₩120K (≈ $90) per purchase; keep only the ads under ₩60K (≈ $45)
The ad in the top row has the highest ROAS, 8.80, but costs ₩121,614 (≈ $91) per purchase. Judged by the absolute number, cost per result, rather than ROAS, it is an ad to cut.
ROAS 8.80 ad
₩121,614
BEP
₩60,000
Bar length is proportional to cost per purchase
9
Supporting diagram · Attribution windows
Supporting diagram
Widen the window from 1 to 28 days: purchases go from 69 to 80
Bar length = counting window (proportional to days). Bold numbers = purchases for the ROAS 2.78 ad / the ROAS 2.91 ad
0 = the ad click (for views, the impression). The window starts at the ad touchpoint, not at the purchase.
Meta official docThe standard attribution settings you pick for an ad set are 1-day or 7-day click, 1-day view and 1-day engaged-view. CourseUses the “Compare attribution settings” columns to put 1-day, 7-day and 28-day click side by side. 28 days is not in the list of official options we read this time.
The longer the window, the more late purchases it catches. The ROAS 2.91 ad stops going from 4 to 5. The 28-day value was read from the comparison column on the course screen, so it is drawn dashed.
Source: Meta Business Help Center, About attribution models and attribution settings · About actions attributed to your ad (checked 2026-11-20)
10
Supporting material · When conversions arrive after a click
Supporting material
Ad-tech data agrees: half of conversions came after day 1, and 13% after 2 weeks
All conversions = 100, split by when they happened after the click. Unit: %
The paper’s point
Look only at data right after the click and you count conversions that haven’t arrived yet as failures. 13% arrive after 2 weeks
The course’s point
Turn it off on 69 at 1 day and you lose the 11 that come later. Look at 1, 7 and 28 days
Both strips show that one day’s numbers are not the whole picture. The late share is smaller for the course ad (14%) than in the Criteo data (about 50%). Criteo’s figure pools display ads from many advertisers; the course’s is one Meta ad, so this ratio varies by product and channel.
Source: Chapelle, O. (2014), Modeling Delayed Feedback in Display Advertising, KDD 2014 (Sec. 2.3 Analysis of the Conversion Delay) (checked 2026-11-20)
11
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
3. Ads we keep even when frequency is high
This screen opens up cost per result by attribution setting and frequency too. Ads #2, #4, #10 and #12 include some with high frequency, yet we keep running them.
On the course screen, the red horizontal boxes mark ads #2, #4, #10 and #12. The middle vertical box is cost per result by attribution setting (1-day, 7-day and 28-day click), and the far-right vertical box is frequency.
12
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
The usual advice: once frequency passes 2, then 3, fatigue has set in, so swap the creative or widen the audience. But a delayed-attribution ad, one that keeps picking up conversions over time, is different. The repeated exposure itself triggers the conversion.
The usual rule
Frequency over 2 or 3 means fatigue. Swap the creative or widen the audience.
Delayed-attribution ad
Repeated exposure drives the conversion. The cost per purchase falls as frequency builds, so don’t turn it off; run it more.
Ads #4, #10 and #12 haven’t built up much frequency yet, but their delayed conversions are large. On screen, frequency is 2.75 for #4, 1.35 for #10 and 1.69 for #12, and #2 at the top is 3.47. For all four, cost per purchase keeps falling from day 1 to 7 to 28.
13
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Frequency and cost per purchase by attribution window, four ads
#2Frequency 3.47
1-day click₩62,638
7-day click₩54,916
28-day click₩53,451
Already performing well. Highest frequency, yet the most efficient.
#4Frequency 2.75
1-day click₩97,755
7-day click₩66,651
28-day click₩63,753
Direct conversions cost ₩97.7K, but this falls to ₩63.7K. Run it longer and it falls further.
#10Frequency 1.35
1-day click₩74,674
7-day click₩37,337
28-day click₩37,337
Falls from ₩74K to ₩37K. Only 2 purchases, so it isn’t precise.
#12Frequency 1.69
1-day click₩74,834
7-day click₩44,901
28-day click₩44,901
Judge the first week on the 1-day number alone and this ad dies along with #4.
14
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
For all four ads, from 1 → 7 → 28 days, the cost per purchase falls
Cost per purchase by attribution window for the four ads (values on screen). At 1 day, all four sit above the ₩60K (≈ $45) BEP. At 28 days, #2, #10 and #12 drop below BEP and #4 falls to around ₩63K.
Below BEP at 1 day0 ads
Below BEP at 28 days3 ads
15
Supporting diagram · Frequency and fatigue
Supporting diagram
Frequency alone isn’t fatigue. Cost per purchase decides
Frequency passed 2 or 3The usual fatigue signal
▼
1 → 7 → 28 days: which way does cost per purchase go?
Falls
Rises
Delayed attribution. Keep it onRepeated exposure works as the conversion trigger (course)Already below BEP: scale up#2 frequency 3.47 · ₩62,638 → ₩53,451Still above BEP: keep watching#4 frequency 2.75 · ₩97,755 → ₩63,753
If CTR also drops: creative fatigueThe more often people see the same creative, the lower the CTR (click-through rate) and the higher the cost per action (Meta developer doc)RecommendedA new ad with different images or video, a wider audience, swapping the creative, or pausing
Source: Meta for Developers, Marketing API Ads Webhooks: Creative Fatigue · Meta Business Help Center, Frequency (checked 2026-11-20)
16
Supporting diagram · Frequency and fatigue
Supporting diagram
The course’s rule of thumb vs. Meta’s doc: fatigue isn’t a frequency number
The usual rule, in the course
Once frequency passes 2, then 3, assume fatigue and swap the creative or widen the audience.
Meta developer doc
Does not define fatigue by a frequency number. It says to check frequency, CTR and cost per action together, daily.
#2 and #4, whose costs are falling, don’t count as fatigued by the doc’s standard either
Source: Meta for Developers, Marketing API Ads Webhooks: Creative Fatigue · Meta Business Help Center, Frequency (checked 2026-11-20)
17
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
4. How many times BEP should you allow?
When judging an ad, it’s fine to watch it up to 2× BEP. Here 2× isn’t a budget; it’s the line for deciding whether to keep the ad. If BEP is ₩60K (≈ $45), watch it up to ₩120K (≈ $90) per purchase. That is the standard I give staff when I hand the work over. When I run it myself and need to be tight, I allow only 1.3×.
Cutoff lines by cost per purchase (BEP ₩60K)
The scale is proportional to the amount. The blue band is the range you watch.
18
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Apply 1.3× to a ₩60K BEP and, using the amount given in the course, you get roughly ₩68K–70K (≈ $51–53). If an ad doesn’t come in under that, you turn it off. The problem: an ad like #10, with direct conversions at ₩74K (≈ $56), dies. Run it a little longer and let frequency build, and it could have dropped to ₩37K (≈ $28). Manage tightly at 1.3× and you lose ads like this.
2× BEPDefault; when staff run it
Watch up to ₩120K → saves the ads that come down over time
1.3× BEPRunning it yourself, tightly
Off if not under ~₩70K → ads like #10 (1-day ₩74,674) die
1.3× or 1.5× BEPWhen money is very tight
Keep repeating the process of making lots of content that adds options
19
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Scale up #2, which is already performing
Ad #2 has high frequency, but its ROAS is a very strong 4.54 and its cost per purchase keeps falling. The usual rule would say swap the creative because of fatigue. But by the attribution-setting numbers, this ad with the highest frequency is the most efficient one. An ad like this gets even better when you scale up its budget.
1Scale up budgetThe pool of people it reaches widens
2Frequency dropsThe same person sees it fewer times
3Keep runningRunning from the lower frequency, it builds back up
4Results improveResults get better as frequency builds
20
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Kill on 1-day ROAS in week one, and…
Turn ads off in the first week based only on 1-day ROAS, and ads #4 and #12 die. As marked on screen, #4’s cost per purchase fell 35% from its 1-day figure and #12’s fell 40%. So I treat up to about 2× BEP as fine. If money is very tight, set it at 1.3× or 1.5×, and keep repeating the process of making lots of content that adds options.
The screen labels #4 “down 35%” and #12 “down 40%”: how far the cost per purchase fell from 1-day click to 7-day and 28-day.
That’s a lot of hands-on tips so far. It may sound unfamiliar to beginners, but anyone who has run ads for a while will nod along.
21
Summary diagram · Cutoff lines and the ads they kill
Summary diagram
Where you cut on day-1 cost decides whether ads that fall near BEP by day 28 die first
Cutoff on 1-day value
Killed on day 1
Their 28-day values
Course 1.3×: ₩68K–70K
#4, #10, #12
₩63,753 · ₩37,337 · ₩44,901
Calculated 1.3× ₩78K, 1.5× ₩90K
#4
₩63,753
2×: ₩120K
None
Watch all four
One arrow = one ad. The course calls 1.3× “₩68K, ₩70K.” But 60,000 × 1.3 = ₩78,000, so both lines are drawn. On the calculated line, #10 and #12 survive and only #4 is killed.
Redrawn from the course content
22
Summary diagram · Scaling up and frequency
Summary diagram
High frequency, strong efficiency: scale up, lower frequency, then let it build again
One dot = one impression for that person. Concept diagram; the number of people and dots are not real values.
The usual rule
Frequency over 2 or 3 means fatigue, so swap the creative or widen the audience
The course’s ad #2
Cost per purchase keeps falling: ₩62,638, ₩54,916, ₩53,451. Keep it on and scale up
Don’t change the creative just because frequency is high. First check that cost per purchase falls across 1-day, 7-day and 28-day click, then scale up.
Redrawn from the course content
23
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
5. Look for a working answer, not the one right answer
Most people want the one right answer. They ask how much to spend a day and how to set up a campaign. That is the problem.
There really is no one right answer. You have to come at it looking for a working answer. Instead of thinking “I should spend ₩N a month on ads” or “CTR is above N%, so it’s good,” look at it through the four points below.
24
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
1
Cost per result Cost per result is king. Grade on an absolute scale before you compare ads with each other. The absolute standard is cost per result, meaning cost per purchase conversion. A cost per result that isn’t a purchase conversion (like the cost per store follow we saw earlier) is not an absolute standard. Compare only the ads that pass the absolute test, and optimization keeps getting better.
2
Consider traffic and conversion conditions As in the boarding cram school case, a high CTR alone isn’t good, and neither is high content engagement like shares alone. Consider what conversion conditions are actually in place.
3
Use cross-metrics Look at the attribution setting, frequency, CPM (cost per 1,000 impressions) and CTR. Read just one metric and you can misread it.
25
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
4
All of this starts with “why” Ask why people acted this way, and what this number really stands for.
One example of cross-metrics is the 3-to-15-second retention rate from the previous chapter. That metric alone can’t tell you whether low retention means the product was so appealing people left to buy it, or they just dropped off. So you look at unique outbound clicks with it.
Without the “why,” you might see only that the first campaign’s 3-to-15-second retention (48.52%) is below the second’s (68.31%), call it a bad campaign and turn it off. Don’t. Truly appealing content can make people skip the full 15 seconds and go straight to buy. Practice asking why a number came out the way it did, and finding the working answer.
26
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
On the screen comparing the two campaigns, 3-to-15-second retention is 48.52% for the first and 68.31% for the second. On that number alone you’d turn off the first, but the first leads on landing-page entry rate (5.82%) and store follows (1,062).
3s → 15s retention48.52% vs 68.31%
Entry rate · store follows5.82% · 1,062
The second campaign has higher retention, but the first leads on entry rate and store follows
27
Supporting diagram · Absolute first, then relative
Supporting diagram
Filter on the absolute test first, then compare only the survivors on cross-metrics
The course’s order applied to the Case 3 screen (11 ads). Bar length = number of ads
Case 3 screenAds in the ROAS-ranked table
11
ROAS 8.80 down to 2.65; one row’s ROAS is off-screen
Screen out ads with high CTR or shares but no conversions (cram school)
▼
3Relative testCompare on cross-metrics
Attribution · freq · CPM · CTR
Keep the ads whose cost falls from 1 to 28 days, and scale them up
4Every step starts with “why”
Why did people act this way? What does this number stand for?
The “why” at the bottom isn’t a step; it’s the question under every step
Source: Meta Business Help Center, Results · Cost per result (checked 2026-11-20)
28
Supporting diagram · Absolute first, then relative
Supporting diagram
Why step 1 uses purchases: results differ by campaign
Meta definition
Result = the number of times your ad achieved an outcome, based on the objective and settings you chose. Results differ by campaign. So the cost per result of a non-purchase result (like store follows in the previous chapter) can’t be the absolute standard in 1.
Source: Meta Business Help Center, Results · Cost per result (checked 2026-11-20)
29
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
6. The working answer: 3 steps, 3 things to know
A working answer means setting a standard, checking it, then adding options, comparing and choosing.
1. Set the standard
How much can you spend per customer?
2. Check the standard
Are you actually spending within that range?
3. Add options, compare and choose
To get within range, and to do better within it: which audience gets which message, and how do you design the way you deliver it?
30
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
To find the working answer, you only need to know three things. The last one, Meta’s algorithm and tools, you’ll know just by taking this course. The other two, audience and marketing funnel, take a lot of practice.
Audience
(= content) · takes a lot of practice
Marketing funnel
(campaign design) · takes a lot of practice
Other
(Meta’s algorithm and tools) · you’ll know it just by taking this course
31
Summary diagram · Three steps to a working answer
Summary diagram
This chapter in three steps: standard = BEP, check = attribution window, options = appeals
1. Set the standard
2. Check the standard
3. Add options, compare, choose
Question to answer
How much can you spend per customer?
Are you actually spending within that range?
Which message, to whom, delivered how?
Example from this chapter
BEP ₩60KCutoff at 2×, ₩120K 1.3× if tight(Sec. 2–4)
1/7/28-day cost per purchase#4 ₩97,755 → ₩63,753 Read with frequency(Sec. 2–3)
4 appeals for one pair of jeansSwitch user and customer ohora, CLUG(Sec. 8–9)
Redrawn from the course content
32
Summary diagram · Three steps to a working answer
Summary diagram
Of the three things to know, audience and funnel take practice
The three things to know for a working answer
Audience (= content)Step 3’s “which message, to whom”Takes a lot of practice
Marketing funnel (campaign design)Step 3’s “delivered how”Takes a lot of practice
Other (Meta’s algorithm, tools)Things like the attribution columns used in step 2Learned just by taking the course
The names of the “three things to know” and which need practice are as on the course screen; pairing them with the steps follows the wording of step 3 in Sec. 6.
Redrawn from the course content
33
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
7. Why fixating on the algorithm, targeting and setup fails
When people start Meta ads, most follow the same path. They hear Meta recently updated to an algorithm called Andromeda, and they go hunting for what it is first. That’s not the way to approach it.
A beginner asks four questions in a row, and as the red text on the course slide says, every one of them is a system view.
34
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
The way of running Meta ads I’m describing is the same one I’ve used since 2014 and 2015. It was rougher early on, but the framework is the same. For more than 5–6 years I ran ads this way: build and run 1-1-1 campaigns (explained later), group them into ABO (ad set budget optimization) so creatives compete, and finally move to CBO (campaign budget optimization).
1-1-1 campaignBuild it and run it
▼
ABOGroup them; creatives compete
▼
CBOThe final stage
35
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
I’ve never used detailed targeting. There are only two exceptions: when an agency or client asks for specific detailed targeting, and when the audience is clearly fixed, such as Chinese or Japanese customers.
Asking what daily budget machine learning needs to optimize is also the wrong view: ₩10K (≈ $7.5) is said to be bad, so must you spend ₩50K (≈ $38) a day? ₩30K per campaign? Same with how to scale up: people say don’t, so is 20% at a time OK? Just assume machine learning starts over if you change any one thing.
36
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Same with how many campaigns to run and how many creatives to make. These questions are all a system view, and people are missing from them. Ask from a human view: why Andromeda came out, why you’d use detailed targeting, why machine learning optimization works and why Meta does it for you, and what mechanism moves when you scale up.
System-view questionRewritten from a human view
What is the Andromeda algorithm? How do I set up detailed targeting?
▶
Why did that algorithm come out? Why use detailed targeting?
What daily budget do I need for machine learning to optimize?
▶
Why does machine learning optimization work? Why does Meta do it for you?
37
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
System-view questionRewritten from a human view
How do I scale up? Is 20% at a time OK?
▶
What mechanism moves when you scale up?
How many campaigns and creatives should I make?
▶
How do I get my audience to respond? What message makes them respond?
Andromeda, seen from a human view
In short, Andromeda means creatives have become very important and targeting has become hyper-personalized. But creatives were always important, and it was always hyper-personalized. Image ads have always worked that way. Say someone with hair loss opens Instagram. Instagram isn’t a place people go with a goal; it’s where they relax and play.
38
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
They’re there to relax, and a post saying “Attention, everyone with hair loss” shows up in the feed. People with hair loss stop. Everyone else doesn’t. Hyper-personalization has worked like this all along: the image content itself splits and decides who will respond. That’s a human view, not a system view. Instead of studying Andromeda, spend the time thinking about how to get my audience to respond, and what message will make them respond.
People with hair loss
Stop scrolling at “Attention, everyone with hair loss.”
People without it
Scroll right past the same post. The content sorts out who will respond.
39
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Three reasons a system approach fails
The reason is simple. However narrow your targeting, a weak message gets no response. Campaign structure and setup are never perfect; there is only optimization. Even if you know how machine learning works, a weak message and weak content leave it no data to optimize on, because machine learning means a machine learning patterns from data on its own. What matters most is whether people respond, and how to make them respond.
40
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Three reasons a system approach fails
1However narrow the targeting, a weak message gets no response
2Campaign structure and setup are never perfect
3Knowing how machine learning works, weak messages and content still leave no data to optimize
What matters most: do people respond?
41
Supporting diagram · Andromeda and human response
Supporting diagram
Andromeda picks thousands of ads from tens of millions, using how people respond
If a message stops no one, there’s no response to feed this loop at all
Source: Engineering at Meta, Meta Andromeda: Supercharging Advantage+ automation with the next-gen personalized ads retrieval engine (2024-12-02) · Meta Business Help Center, About ad auctions (checked 2026-11-20)
42
Supporting diagram · Andromeda and human response
Supporting diagram
What the course and Meta’s own documents say
Course
Creatives were always important and always hyper-personalized. The content sorts out who will respond
Engineering at Meta blog
The retrieval system, the first stage of ad recommendation. After launch: recall +6%, ad quality +8% in some segments
Meta auction doc
The winner is the ad with the highest total value, combining bid, estimated action rate and ad quality
Source: Engineering at Meta, Meta Andromeda: Supercharging Advantage+ automation with the next-gen personalized ads retrieval engine (2024-12-02) · Meta Business Help Center, About ad auctions (checked 2026-11-20)
43
Supporting material · Paying for audience segments
Supporting material
Brokers’ “male” segments were only 42% male on average, no better than random
Share of people in a segment who actually have that trait, %. Gray dot = one vendor, blue dot = average, dashed line = random pick
CourseI’ve never used detailed targeting. However narrow the targeting, a weak message gets no response CautionThis study measured segments sold by third parties for web ads. It did not measure Meta’s detailed targeting.
Adding the buying platforms’ optimization lifts it to 59%, but vendors ranged from 40% to 72%. The paper concludes that such targeting often isn’t worth its extra cost.
Source: Neumann, N., Tucker, C. E. and Whitfield, T. (2019), How Effective Is Third-Party Consumer Profiling and Audience Delivery? Evidence from Field Studies, Marketing Science 38(6), forthcoming version (MIT IDE, Tables 4–6) · DSpace@MIT abstract (checked 2026-11-20)
44
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
8. Same product, different appeals
The same product is used differently by different people. One pair of pants can carry four messages.
The course slide shows four messages for one pair of pants, with the conclusion beneath: “What you need to change is the appeal, not the setup.”
45
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
For the officeComfy, a bit formal
Sitting at work for hours gets uncomfortable. Here are jeans you can wear to work that look sharp and feel easy.
For datesAttract a partner
Wear these and women, or men, will love them.
Slimming lookCovers the figure
If you have thick thighs, these make your thighs look slimmer.
Longer-leg lookHeight boost
If you have short legs, these make you look taller and better proportioned.
The jeans are the same, yet every message differs, because people buy a product for different reasons. Porsche is the same.
Second carBought to look at and admire
Showing offBought to be seen in
Business toolBought as leverage for a business
46
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
So what you change first is the appeal, not the setup. Never forget that in Meta ads, content carries 70% of the weight. When a winning creative suddenly dies, the easy way to see it is that the pool of people who responded to that creative, that message, is used up. Vary the appeal and the same product keeps getting responses.
1Winning creativePeople who respond to one appeal buy
2Pool used upEveryone who’d respond to that appeal is gone
3Creative diesThe winner’s results suddenly drop
4Change the appealPeople with other reasons respond to the same product
47
Supporting diagram · Each appeal has its own pool
Supporting diagram
Even one pair of jeans has a separate pool for each reason to buy; when one runs dry, switch appeals and move on
Porsche is the same
Second car (to admire)
Showing off (to be seen)
Business leverage tool
Meta developer docFatigue rises the more often people see the same creative. Its fixes include a new ad with different images or video, and a wider audience.
The course explains a creative’s sudden death as pool exhaustion and says to change the appeal, not the setup. Concept diagram; pool box sizes and fill are not real values.
Source: Meta for Developers, Marketing API Ads Webhooks: Creative Fatigue (checked 2026-11-20)
48
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
9. Widening the pool by switching user and customer
Well-known examples are ohora (a Korean gel-nail brand) and CLUG (a Korean massager brand). First, separate the user from the customer.
User
The person who uses the product
Customer
The person who buys it, the purchaser
ohora was used mostly by women in their 20s and 30s as gel nails. Once messages for this audience ran dry (whether they truly ran dry, we can’t know), they changed the message. More precisely, they expanded it: “Mom’s hands are lovely; make Mom’s hands pretty.” They kept the buyer the same and changed the user to drive purchases.
49
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
CLUG did the same. First it featured exam students, so parents would buy it. Next it got people in their 20s and 30s to buy it themselves as a wireless massager. After that it pitched it as a gift for parents, to be used by the parents. I read this as switching user and customer again and again because the pool kept running out.
In the course’s ad examples, ohora (“My hands still look pretty good, right?”) and CLUG (“When you want to show you care, give CLUG”) both moved the user to the parents’ generation.
50
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Brand · stage
User (who uses it)
Customer (who buys)
ohora start
Women in 20s–30s
Women in 20s–30s
ohora expand
Moms
Unchanged
CLUG start
Exam students
Parents
CLUG next
20s–30s
20s–30s
CLUG third
Parents
People buying a gift for their parents
This happens a lot in real work. So the claim that Meta ads succeed only if you crank out endless content is flat wrong. The key is how sharp and clear a message you can pull out, and whether your audience responds to it. What machine learning actually is, the thing so many people wonder about, comes in the next chapter.
51
Supporting diagram · Who uses it and who buys it
Supporting diagram
ohora and CLUG: same product, new user and buyer
ohora1Start · gel nails, women in 20s–30s → 2Expand · make Mom’s hands pretty. The buyer stays the same CLUG1Start · for exam students, parents buy → 2Next · wireless massager → 3Third · a gift for parents, parents use it
Ansoff matrixIn its terms, keeping the product and changing the market is “market development” (secondary source). CourseSees the reason for repeatedly switching user and customer as pool exhaustion. The instructor says this is his own analysis.
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Wrap-up
Action checklist
In Meta’s data dashboard, search “attribution setting,” add it as a custom metric, and look at results and cost per result for 1-day, 7-day and 28-day click.
Set your product’s BEP and note how many times BEP each ad’s cost per purchase is. Turn off ads above 2×; for the ones within it, check whether cost falls across attribution windows.
Don’t kill an ad the moment frequency passes 2 or 3. First check whether its 1-day, 7-day and 28-day cost per purchase is falling. If it is falling and already efficient, scale it up.
53
Part 1 · 1-2. Keep or kill: why we scaled the 278% ROAS ad, not the 291% one
Wrap-up
Action checklist (continued)
Don’t conclude from one metric. Cross-check attribution setting, frequency, CPM, CTR and unique outbound clicks, and write down “why did people act this way?”
For one product you sell now, write four appeals with different reasons to buy. Also make one message that puts the user and the customer in different places.
54
That is the end of the free chapters. The next 11 chapters and both appendices: scaling 30x in 4 days, 1-1-1 / ABO / CBO, why efficiency breaks, funnels and bottlenecks, and the 2-axis content matrix with real cases.