Rules
Case study genres: five blind spots worth knowing about
Campaign case studies come in five genres with different purposes and different blind spots. Knowing which one you are reading tells you what it can support.
Not all case studies are the same document. An award entry, a vendor page, a trade press write-up, a conference talk and an internal debrief are five different genres with five different authors, five different audiences and five different sets of things they are allowed to leave out.
Identify the genre first. It tells you more about what the document can support than anything inside it. The reading method that follows from this is in campaign case studies.
What to take away
- The genre determines what was omitted, and what was omitted is usually the thing you wanted to know.
- Only two of the five genres have any incentive to describe a failure, and one of those is your own record.
- A case study is evidence about mechanism and rarely evidence about effect.
The five genres
| Genre | Written for | Optimized for | Systematically omits |
|---|---|---|---|
| Award entry | A jury, against criteria | A narrative arc with a measurable ending | The budget, the failures, and everything outside the winning metric |
| Vendor or agency page | A prospective buyer | Demonstrating capability | Comparable campaigns that did not work |
| Trade press article | Readers, quickly | Novelty and a quotable line | Method, denominators, and what else was running |
| Conference talk | An audience in a room | Memorability | Detail, because detail does not survive a slide |
| Internal debrief | Your future self | Being useful later | Nothing, if written honestly, which is the point |
The first four all share one property: they exist to persuade someone of something. That does not make them false. It makes them selected, and selection is the harder problem, because there is nothing in the document to disagree with.
Five case study genres
Genre
- Award entry
- Jury, criteria
- Vendor page
- Prospective buyer
- Trade press
- Readers, quickly
- Conference talk
- Room audience
- Internal debrief
- Future self
Written for
- Award entry
- Narrative arc
- Vendor page
- Capability
- Trade press
- Novelty
- Conference talk
- Memorability
- Internal debrief
- Useful later
Optimized for
- Award entry
- Budget, failures
- Vendor page
- Failed campaigns
- Trade press
- Method, denominators
- Conference talk
- Detail
- Internal debrief
- Nothing, if honest
Omits
- Award entry
- Vendor page
- Trade press
- Conference talk
- Internal debrief
What each genre is genuinely good for
Award entries are a good source of ambition and structure. They show what a team thought was worth attempting and how they framed it. Read them for the shape of the idea and ignore the results section.
Vendor pages are the best available description of how a service actually operates. Skip the outcomes and read the mechanics: what was delivered, in what order, who did what.
Trade press is a good early-warning system for what is changing in the market and a poor source of evidence about anything. Read the quotes, not the numbers.
Conference talks are frequently the most honest of the four, because a speaker in a room tends to say the interesting part out loud in a way they would not write down. The questions afterward are worth more than the talk.
Internal debriefs are the only genre that can tell you what actually happened, and almost nobody writes them.
The selection problem, stated plainly
Every published case study worked well enough to publish. The ones that did not are not on a list somewhere; they simply do not exist as documents. A body of case studies samples successful campaigns, not campaigns overall, so typical results from it come from a filtered set.
That is survivorship bias, and it makes case studies systematically optimistic about everything.
The same mechanism operates inside your own organization. Wins get written up and disappointments get discussed once and forgotten, which is publication bias with a smaller audience.
Reading any of them: the same five questions
- Who wrote it and what do they sell?
- What was the comparison? If it is the previous period, it is a description.
- What is the denominator, and what is excluded from it?
- Was the window chosen before or after the results were seen?
- What is not mentioned at allbudget, other channels running at the same time, the campaigns from the same period that are not here?
The last one is the most productive and the least used. A case study that describes a campaign in detail and never mentions the rest of the marketing running that quarter has left out the main competing explanation.
Five questions for any case study
- Who wrote it and what do they sell?
- What was the comparison?
- What is the denominator, what is excluded?
- Was the window chosen before or after?
- What is not mentioned at all?
Where those questions have solid answers you have something better than a case study, which is a measurement, and the methods for that are in measurement and ROI.
What a case study is genuinely evidence of
It shows that something was possible: this format, in this category, with this kind of partner, produced something worth writing about once. That is real information, and it is what published cases should be read for.
It also shows how a deal was structured, which is often stated openly because it is not the sensitive part. That is useful when you are designing your own, and it connects directly to what you should be negotiating, in rates and negotiation.
What it is not evidence of: what results you should expect, what a fair price is, or whether a partner will work for you. Those depend on your product, your offer, your audience and your execution, none of which are in the document.
Writing one yourself
If you are producing a case study for external use, three things make it more credible without giving away anything commercially sensitive. State the comparison you used. State the window and when you chose it. Say one true thing that did not work. The third costs almost nothing and it is the fastest way to be believed.
If you are writing an internal one, write the version that would be useful to someone who has to make the same decision in a year, including the parts that reflect badly on the process. The material for it comes out of the brief and the delivery record, which is why campaign briefs is where the useful detail is generated.
Bottom line
Identify the genre before reading the content: award entry, vendor page, trade press, talk or internal debrief. Read the first four for mechanism and ambition, not for expected results, because they are a sample of successes rather than a sample of campaigns. Ask the same five questions of all of them, and write the internal version nobody else will.
Common questions
Are vendor case studies worthless?
No. They are the best description available of how a service works. The results section is the part to discount.
How do you find case studies of failures?
Mostly you do not, publicly. You can ask directly in a room, and people answer far more often than they write it down.
Should a brand publish its own?
If it helps recruit partners or explain your standards, yes. Include one thing that did not work and the document immediately becomes more useful to everyone including you.
Is an award-winning campaign a good model?
It is a good source of ideas and a poor source of expectations. Award criteria reward a narrative, which is not the same as rewarding an effect.
What makes an internal case study worth keeping?
That it records what happened rather than what was reported, including the changes mid-flight and the numbers that were set aside.
