Lesson 2.30: Benchmarking against peers without fooling yourself
'What do companies like us spend, and how do we compare?' is one of the first questions any board asks a new CISO. Benchmarks help, but they mislead easily: peers count incidents differently, companies differ in size, and the ones with bad years often don't answer surveys. In the lab you compare Northwind with four peers, normalise budgets by revenue and by employee, compare incidents only where definitions match, read the survey's response notes and write a summary that a board can trust.
18 min
You will learn to
- Normalise figures by size (revenue, employees)
- Spot definition mismatches between peers
- Compare only like with like
- Recognise survivorship and response bias
- Write a benchmark summary with its caveats
Key terms
- Normalisation
- Dividing by size (revenue, employees) so different companies can be compared.
- Definition mismatch
- Two sources counting the 'same' thing differently.
- Survivorship bias
- Drawing conclusions only from those who remain visible, e.g. companies without a breach.
- Response bias
- Distortion because those who answer differ from those who don't.
- Sanity check
- A quick comparison that flags obvious outliers, not a target.
← Keeping a policy framework alive · All lessons · Command line arguments and options →