Agency reviews and rating methodology

An agency review is useful when readers can see what was assessed, which evidence supports the judgement and where that evidence stops. A rating requires a stated method applied consistently. A prominent listing, polished case study or customer star rating is not, by itself, an editorial assessment of agency quality.

Define the assessment

State whether the review examines publicly described services, work from a past engagement, a proposed team, customer accounts or something else. Name the services, market and assessment date. If no service was purchased or tested, say so. A review based on public information must not read like a first-hand customer account.

Explain how agencies qualify for inclusion. A limited set may help a reader explore options, but it should not be presented as a complete survey of Australian agencies. State material exclusions and any gaps in the available evidence.

Key Steps in Conducting an Independent Agency Review

Apply common criteria

Choose criteria that answer the question the comparison promises to address, then use them for each agency:

Record where each material claim came from, when it was checked and what it establishes. Distinguish agency statements, customer accounts and independent records. Let an agency clarify a material factual point while the publisher keeps the editorial judgement.

Apply editorial standards

For publications within its jurisdiction, the Australian Press Council’s Statement of General Principles is binding. It says publications should take reasonable steps to meet its standards regardless of how content is produced, including when AI is used.

Keep factual material distinguishable from opinion. The Press Council’s principles require factual material to be accurate and not misleading, and opinion not to rest on significantly inaccurate facts or omitted key facts. A rating is an editorial judgement, but the facts supporting it should remain identifiable.

The Press Council says significantly inaccurate or misleading published material should receive a correction or other adequate remedial action. Where material refers adversely to a person, it calls for a fair opportunity to publish a reply when reasonably necessary to address a possible breach of its fairness and balance principle.

The same principles say conflicts of interest should be avoided or adequately disclosed, and should not influence published material. Material gathered by deceptive or unfair means should not be published unless doing so is sufficiently in the public interest.

Key Regulatory Bodies and Their Roles in Review Integrity

Decide whether to rate

Publish the criteria, any weights, the treatment of missing information and the assessment date before readers rely on a score. Show the observations behind it. An unanswered question is neither evidence of poor performance nor grounds for full credit. If essential evidence is missing, an unscored profile with a clear limit may serve readers better than a precise number.

Keep customer star ratings separate from editorial scores. If customer ratings appear, readers need context: the number of reviews, and any incentives or commercial relationships that may affect what they see. If a method changes, identify which profiles used each version and whether older ratings were reassessed.

Customer Star Ratings vs. Editorial Scores

Put limits beside conclusions

A case study may concern a different team or project. A public service list may show an offer without demonstrating delivery. Place those qualifications near the summary, score or table entry they affect. Explain the buyer and work the comparison covers so a reader can judge its relevance to their own brief.

Label paid placements where they appear and explain any effect payment has on inclusion, order or presentation. Keep paid visibility distinct from an editorial judgement. Provide a way to report factual errors, check new evidence and explain material corrections to readers.

The Australian Competition and Consumer Commission accepts reports about fake or misleading online reviews and review manipulation, and about possible false or misleading claims. It uses reports to inform education, compliance and enforcement work, and may investigate and take compliance or enforcement action; it can also require businesses to back up claims about their products or services.

The ACCC does not resolve individual disputes about online reviews or misleading claims, and does not provide legal advice. Its role is distinct from an editorial review: a rating should show readers its own evidence and reasoning rather than imply the regulator has verified the assessment.