Capability Checks
Part of Checking agency experience
Asking reference customers useful questions
Prepare an agency reference call that tests scope, delivery, problem handling and the experience of working with the team.
Ask a reference customer about work they observed directly: the scope, the people involved, how decisions were made and what happened when delivery became difficult. Specific examples reveal the agency’s fit with your proposed work better than a general recommendation.
Prepare a relevant conversation
Identify the condition you most need to check: working with several approvers, managing a technical dependency, or responding to disappointing results. If the customer agrees, ask the agency to introduce you to a customer whose engagement involved that condition.
Briefly explain the work you are considering and the expected length of the call. Ask a core set of questions across candidates, then follow up on relevant details.
Note the reference’s role, when the work occurred and how closely they worked with the agency. Someone who approved invoices may have seen different things from someone who worked with the delivery team each week.
Ask for events, not adjectives
Begin with open questions:
- What was the agency hired to deliver, and what did your team provide?
- Which agency people did you work with regularly, and what decisions did they make?
- What did the team do well? Can you describe an instance?
- What became harder than expected, and how did the agency respond?
- How were agreed dates, outputs and review steps handled when plans changed?
- What happened when you disagreed with a recommendation or requested a correction?
- What would you clarify earlier if you engaged that agency again?
If the reference mentions a result, ask what was measured and what other work was happening then. The call can be useful without confidential performance data. Their account of the agency’s actions, and the limits of what they observed, may tell you more.
Interpret what the reference could see
An agency-selected reference may describe a favourable experience. Take it seriously, but remember one engagement is not a general performance record. Ask whether the reference worked with the people proposed for your account and whether their scope and client-side resources resembled yours.
Make broad descriptions concrete. If the reference calls the agency responsive, did that mean a quick acknowledgement, a completed fix or a changed plan? If they would hire the agency again, for what kind of work? Mark a question as unknown when the reference could not observe the answer.
Where their account differs materially from the proposal, ask the agency to explain. A changed team or method may have a reasonable explanation; check how the proposed arrangement addresses your concern.
Use the call in your decision
Record what the customer observed, what it suggests for your project and what still needs confirmation. Compare those notes with the proposed scope and team, then ask the agency focused follow-up questions. A reference call is most useful when it tests a risk in your engagement without claiming more than one customer could know.
