Incrementality measurement is often used to settle a budget argument, yet many tests only show that a platform recorded conversions. A defensible review defines who or what received the treatment, what comparison is valid, when the effect should appear, how interference is controlled, and which mature outcome matters. The mistakes below help a small team find where a result stops being causal evidence.
1. Mistake: calling attribution lift
A platform can assign credit to an ad without proving that the ad created an additional outcome. Write the causal question in one sentence: what happened because of the intervention that would not have happened otherwise?
Keep attributed conversions, observed conversions, incremental conversions, accepted work, delivery, and mature value in separate rows. A difference in credited totals is not automatically a difference caused by spend.
2. Mistake: changing the treatment mid-test
Record campaign, audience, geography, budget, bid, creative, landing page, offer, form, and measurement versions. If several variables change at once, note that the test can identify a bundle, not one tactic.
Freeze the treatment where possible and create a change log. If the business must change a setting for safety, mark the interruption and restart or narrow the interpretation rather than stitching incompatible periods together.
3. Mistake: using a weak comparison
Name the control or baseline, assignment rule, eligibility, exclusion, time window, and expected balance. A previous week, an unexposed audience, a matched geography, and a holdout answer different questions.
Check seasonality, supply, sales coverage, policy, brand activity, competitor events, and spillover. If the comparison differs on a material driver, label the result observational and keep a causal claim open.
4. Mistake: ignoring source and event scope
Use GA4 traffic-source dimensions to document user, session, event, and campaign scopes. Scope explains how a source is attached to a record; it does not establish incrementality.
Check whether consent, browser behavior, cross-domain paths, direct traffic, or source loss affects treatment and comparison differently. A measurement gap that is uneven across groups can bias the result before any model is applied.
5. Mistake: optimizing to a proxy
Define the outcome hierarchy: valid event, unique lead, accepted lead, opportunity, booking, delivery, margin, or cash. Use the shortest valid lag for the decision, but do not call a proxy mature value.
For paid media, the Google Ads conversion-data guidance is a platform boundary for conversion reporting. It does not provide a local test design or prove that a recorded conversion was incremental. If a mature outcome is imported after the initial event, the offline conversion import guidance helps record matching, timing, identifier, consent, and upload status without turning a platform report into a causal study.
6. Mistake: ignoring lag and interference
Record first exposure, conversion, acceptance, booking, delivery, and payment dates. Define the observation window and late-conversion rule. A short window can undercount slow outcomes; a long window can mix a new treatment with later changes.
Check retargeting, shared accounts, household or company exposure, organic spillover, sales outreach, and cross-channel substitution. If control people can receive the treatment through another route, preserve the interference as a limitation.
Write a causal diagram before choosing a statistic. List treatment, mediators, external drivers, exclusion paths, and the outcome. The diagram can be simple, but it should reveal when the measurement is trying to control for a variable created by the treatment or omit a commercial dependency.
7. Mistake: overclaiming a small sample
Report population, eligible units, treatment count, comparison count, exclusions, missing data, uncertainty, and decision threshold. Avoid a universal confidence phrase when the design, variance, or sample is not comparable.
Use a pilot to learn whether the assignment, data joins, timing, and outcome definition work. A null result can mean no lift, insufficient power, poor treatment, broken measurement, or a lag that has not matured. Keep a diagnostic row for eligibility, assignment, exposure, event capture, CRM join, outcome maturity, and interference.
8. Use the mistake log
| Mistake | Evidence | Risk | First correction | | — | — | — | — | | attribution called lift | platform credit only | causal overclaim | rewrite the question | | treatment changed | version log has many edits | effect is bundled | freeze or restart | | comparison is weak | imbalance or spillover | biased estimate | narrow the claim | | scope differs | source/consent gap | uneven measurement | reconcile sample | | proxy is mature value | click or lead only | budget error | split outcome layers | | lag ignored | late stages missing | premature conclusion | set cohort cutoff | | sample overclaimed | unstable estimate | false certainty | pilot or hold |
Record owner, date, affected campaign, correction, expected evidence, and release condition. Add a reviewer who is not responsible for the treatment so adverse evidence is challenged before a budget decision. Keep raw assignment, exposure, event, CRM, and outcome files so a summary can be reproduced after a tracking correction or late import.
9. Close with one bounded claim
Choose CAUSAL_EVIDENCE, OBSERVATIONAL_SIGNAL, MEASUREMENT_REPAIR, PILOT, or HOLD. State what the result does not prove. Preserve treatment, comparison, raw data, change log, and rollback.
Name the next review date and the evidence threshold for changing the claim. Keep that threshold explicit in the decision record. Review it with the owner before changing spend.
Incrementality work is credible when the design survives a skeptical question about treatment, comparison, timing, interference, and mature value. The mistake log protects the budget decision from a persuasive attribution graph that cannot answer the causal question.
How did this article land?
Choose one reaction. You can change it anytime.
