Case study standards
The Playbook commits to named cases over anonymous “a major retailer” framing. Every Section 5 anchor case — and every supporting case named elsewhere in a chapter — has to clear the bar below before it enters prose. This page is the bar.
The verified register itself — every entry, with the company named, the source linked, the date window stated, and the caveat the chapter applies — sits in the subscriber library alongside the Playbook bundle.
What every entry has to be
A case study earns its place only when all six rules hold. If a candidate misses any of them, a different case is used or the claim is cut.
| Rule | What it means |
|---|---|
| Named company | Not “a major retailer”, not “a Fortune 500 SaaS firm”. Named. |
| Named product, team, or campaign | Specific enough that you could find the case independently. |
| Date window | ”Q2 2024”, “mid-2023”, “2016 pilot to 2019 enterprise deal” — bounded. |
| Specific metric with unit | ”Reduced brief-to-draft cycle by 58%” beats “dramatically faster”. Comparator included where published. |
| Retrievable primary source | Company blog, press release, conference talk, published interview, regulatory filing, tribunal record. Not a vendor landing page that might be removed. |
| Origination flagged in prose | If the case was published by the vendor supplying the tool, the chapter says so. You weigh vendor-originated cases differently and you are entitled to. |
Origination — three tiers, weighted differently in prose
Not every case carries the same evidentiary weight. The register splits them into three tiers, and the prose treats each tier accordingly.
- Independent. Third-party legal adjudication, regulator finding, peer-reviewed paper, or trade-press reporting where no vendor co-authored the account. This is the highest verification class.
- Partial vendor. Vendor contributed quotes or framing, but at least one independent outlet has reported the same facts and figures.
- Vendor-published. The case lives on the vendor’s own marketing surface. Useful as a directional signal of what the technology can do; not useful as a forecast of what your team will achieve. The chapter says so each time.
Headline figures from vendor-published cases are described as ceilings, not medians.
How each entry is verified
Every register entry passes the same three-step check before it enters chapter prose.
- Resolve the primary source. Open the URL. Confirm the page renders, the publication date is visible, and the headline figure on the page matches the figure cited in the chapter.
- Corroborate where possible. For vendor-published cases, look for an independent outlet (trade press, customer story, conference talk) that reports the same facts. Note the corroboration on the entry.
- Record the retrieval date. Every entry carries a
Verifieddate. Sources are re-checked when a chapter ships and on a rolling annual cadence after that. If the URL no longer resolves, the entry is downgraded topending-verificationand pulled from prose.
The same three-step check is the procedure a reader can run to audit any claim in the Playbook.
Worked examples — the standard in practice
Two entries are shown publicly so the standard is visible at the case level, not just the rule level. The remaining entries — and the full register — read in the subscriber library.
Air Canada (Moffatt v. Air Canada) — Chapter 15
| Origination | Independent. Third-party legal adjudication is the highest verification class on the register. |
| What happened | The Civil Resolution Tribunal of British Columbia held Air Canada liable for negligent misrepresentation by its customer-service chatbot. |
| Result | Tribunal awarded CAD $812.02 in damages. The damages quantum is small; the precedent is what the case carries — the first Canadian tribunal ruling explicitly rejecting the “AI as separate legal entity” defence. |
| Window | Incident November 2022; tribunal decision 14 February 2024 (Moffatt v. Air Canada, 2024 BCCRT 149); chatbot taken offline by April 2024. |
| Primary source | CanLII tribunal record · Dentons Data legal analysis (corroborating) |
| Caveat in prose | The procedural mechanism (CAD $812.02 in BC) is jurisdiction-specific. The principle (the organisation is responsible for AI output) is jurisdiction-agnostic. The Playbook frames the case explicitly. |
CarMax × Microsoft — Chapter 3
| Origination | Vendor-published, with independent corroboration. The vendor (Microsoft) authored the customer story; CIO Magazine reported the timeline and the 80% figure independently. |
| What the team did | Synthesised 100,000+ customer reviews into 5,000 make/model/year car research pages using Azure OpenAI Service (GPT-3). |
| Result | A few months of work versus an estimated eleven years of manual production. 80% editorial review approval rate on AI-generated text after fine-tuning. |
| Window | Case published 24 May 2022; implementation preceded publication. |
| Primary source | Microsoft customer story · CIO Magazine corroboration |
| Caveat in prose | The model is GPT-3, not GPT-3.5, despite some secondary-source confusion. The 80% approval rate is on a bounded task class (summarising verified customer reviews). Less bounded tasks land lower. |
What does not earn an entry
These categories are rejected by default. They never become anchor cases and they do not appear in the register.
- Anonymous cases. “A major retailer”, “a Fortune 500 SaaS firm”. You discount anonymous cases, correctly.
- Vendor case studies with no disclosed methodology. Unverifiable; likely best-case selection.
- Aggregator summaries. Blogs summarising other blogs without a primary-source link.
- Forward-looking cases pending verification. Held until a primary source resolves. They do not enter chapter prose and they do not appear in the register.
- AI-generated claims. Every figure traces to a human-authored, dated source.
How to flag a case
If a primary source no longer resolves, a tribunal record has been amended, a vendor case has been retracted, or the company itself has restated the figures — flag it through the errata channel. The seven-day acknowledgement and sixty-day-fix SLAs apply to register entries the same way they apply to chapter prose.
If you subscribe, the register itself reads in your library.