Module 4 of 8 · 25 minutes
Recognize variation before claiming change
Explain why one answer is not a trend or a causal result.
Answers can differ when we repeat the same question. Engines also differ from one another. A change between two screenshots may reflect ordinary variation rather than the effect of a website edit.
Compare like with like
Preserve question wording, class, engine, category, date and method. Keep failed or excluded observations visible with reasons. A new question set and a different engine cannot silently become the next point in the same trend.
Mixed evidence belongs in the report
If two repeated answers name a company and one does not, show that disagreement. A single best answer is not a representative sample. Name and citation counts can move in different directions. Summarize both and keep the underlying receipts.
Observation is not causation
An increase after an edit is consistent with several explanations: the edit, new external evidence, engine changes, timing or sampling variation. A careful report states what changed, what stayed comparable and what remains uncertain. It does not promise that a tactic caused a ranking or revenue gain.
Practice, then check
Use five minutes to sketch three synthetic repeated runs: 4/10, 6/10 and 5/10. Spend five minutes deciding what one later 6/10 result can establish. Read the engine-divergence study for ten minutes. In the final five minutes, write a before/after claim that preserves uncertainty.
Your learning artifact
Make a tiny three-run table and a two-sentence limitations note. Do not run paid engine calls or test a real customer for this course.
No real prospecting, paid measurement or company deliverable is required.