Renewal Decisions
Part of Agency reporting and accountability
Asking an agency to explain uncertainty rather than hide it
Ask an agency to distinguish observations, interpretations and reporting limits, then agree when to act or review again.
Ask the agency to separate what the report shows, what might explain it and what remains unknown. This gives the client a basis to decide whether to act, gather more evidence or set a review date.
Put the limit beside the claim
When a report says performance improved, ask for the metric definition, comparison period and size of the change. Then ask: “What else changed at the same time?”, “How recent is the data?” and “What evidence would change your interpretation?” Put the main limit beside the claim it affects, where a decision maker can use it.
A short note can cover four fields:
Field / What to state
- Observation
- The measured change and its source
- Interpretation
- The leading explanation and plausible alternatives
- Limit
- Missing data, reporting delay, measurement rule or external change
- Decision
- What to do now and when to revisit it
Suppose a hypothetical report shows fewer enquiries this week than last. That alone does not establish that a campaign failed.
If the measure relies on Google Ads conversions, later customer actions may change recent totals. The agency can label the comparison provisional, check which conversion actions were counted and set a review date. It should still flag any operational issue needing prompt attention.
Make each qualification specific
“Data is imperfect” tells the client little. For Search Console or GA4, name the specific data-quality condition only when it is confirmed for the figure in question. GA4 can also include modelled key events where direct observation is unavailable. Check whether a confirmed condition applies to the figure and whether it could affect the decision.
Ask whether a limit could change the agency's recommendation. If it could, consider a smaller action, another check or a later decision. If it could not, record why. Avoid a precise confidence percentage unless there is a defensible method behind it.
Return to the claim
At the next review, compare the earlier explanation with what became known. Did the provisional figure settle? Was an alternative explanation checked?
Did the recommendation change? Keep a record of the original claim and its revision so the next decision uses the best available evidence.
