Capability Checks
Part of Agency reporting and accountability
Distinguishing agency activity from business impact
Separate delivered work, exposure, response and business outcomes so an agency report does not overstate its impact.
Agency activity is what the team did. A business outcome is what the organisation observed. Business impact makes a stronger claim: that the agency's work contributed to a change that would otherwise have differed. A useful report lays out the steps from work to outcome, and does not treat a task count or attributed conversion as proof of impact.
Label each measure
Use four levels when reading a performance update:
- Activity:work completed, such as publishing a page or changing a campaign setting.
- Exposure:an opportunity to encounter the work, such as an impression.
- Response:an observed action, such as a click or submitted form.
- Business outcome:the result the client agreed to pursue, such as an accepted enquiry, completed order or retained customer, using the client's definition.
Completed work is evidence of delivery even when an outcome has not appeared. A rise in response can justify investigation without being called revenue. Ask the agency to label each headline number at the level it measures.
Consider a hypothetical service business whose agency publishes new pages and reports more search clicks. Publishing is activity; clicks are response.
To examine the business outcome, the client would need an agreed record of relevant enquiries and what happened to them. Search Console counts clicks on links from Google under its reporting rules. Those clicks are not interchangeable with site sessions, enquiries or sales.
Four Levels of Performance Measurement
- ActivityWork completed, such as publishing a page or changing a campaign setting.
- ExposureOpportunity to encounter the work, such as an impression or search click.
- ResponseObserved action, such as a click, form submission, or site visit.
- Business OutcomeAgreed result such as an accepted enquiry, completed order, or retained customer.
Search Clicks vs. Business Outcomes: What's the Difference?
- Search Console ClicksCount of clicks on Google search results; not equivalent to site sessions, enquiries or sales.
- Client-Defined Business OutcomeAccepted enquiry, completed order, or retained customer — based on client’s internal records.
Check the explanation for a change
A before-and-after chart gives context, but timing alone does not identify a cause. Ask whether the offer, stock, pricing, sales process, season or other marketing changed during the period. The agency should describe plausible alternatives and match its conclusion to the evidence available.
Google Analytics attribution assigns credit across interactions on the path to a key event. It can describe recorded paths under the chosen reporting method. It cannot, by itself, show how many outcomes would have happened without the agency's work. If a decision depends on that stronger claim, agree on an appropriate evaluation approach and its practical limits before estimating impact.
Using Attribution Models to Assess Impact
- ProsShows paths to key events using chosen reporting method; helps understand user journeys.
- ConsCannot prove what would have happened without agency work; cannot measure counterfactual impact alone.
Keep measures that support decisions
Use activity measures to confirm delivery or diagnose a problem, response measures to examine audience behaviour, and the business record to assess the agreed outcome. Show gaps in that record. For each headline, ask what decision would change if the number rose or fell. If there is no answer, the measure may belong in a diagnostic appendix rather than the main account of performance.
Key Measures for Decision-Making
- Use Activity Measures For
- Confirming delivery or diagnosing problems.
- Use Response Measures For
- Examining audience behaviour (e.g., clicks, form submissions).
- Use Business Records For
- Assessing agreed outcomes like sales or customer retention.
