What If We're Measuring the Wrong Thing?

A quiet safety dashboard can feel like success. Is it measuring the right thing, or just the absence of failure?

It's 2am….again!

You're not asleep. You're looking at last month's safety dashboard in your head, the one that went green across the board. Zero incidents. Zero near-misses reported. Zero open findings past due. On paper, everything is fine. Cause for a good night of sleep…right??!!

So why does it feel like nothing is fine?

You tell yourself the numbers are good. You know your personnel are experienced, careful and well trained. You know the aircraft went out on time, every time, all month and yet here you are, wide-awake, running through the shift handovers in your mind, wondering what (if anything) you missed.

Maybe you didn't miss anything, or maybe the dashboard is telling you the wrong story, because it was built to measure the wrong thing.

The comfort of a quiet month

A quiet month feels like success. No incidents means nothing went wrong. Nothing went wrong means the system is working. That is how most safety reporting is built, counting backwards from failure.

Lagging indicators, incident counts, near-miss numbers, audit findings, all describe what already happened. They tell you the system failed or that it hasn't failed yet. They cannot tell you how close it came, how tired the workforce is right now, or how many small workarounds are quietly holding the operation together tonight.

A safety system built entirely on lagging indicators can look perfectly healthy right up until the month it isn't. You may perhaps also wish, that there were some small failures reported. That way you would achieve a peaceful night of sleep, comfortable in the knowledge that your safety management system works effectively.

What the dashboard doesn't ask

Ask yourself honestly. Does your current reporting tell you:

How many of your personnel raised a concern this month, and felt heard afterwards?
How long a corrective action actually takes to close, not just whether it eventually closed achieving and/or re-establishing the correct safety level or standard?
how often the work is done differently from how the procedure describes it?
whether people genuinely believe that safety reporting changes anything?

Most dashboards were never built to answer these questions. They were built to satisfy an auditor, not to describe the safety health of the organisation underneath the numbers.

Outcomes are not the same as conditions

There is a difference between measuring outcomes and measuring conditions:

An outcome is what happened, an incident, a delay, a finding.

A condition is what allowed it to happen, or what is quietly allowing the next one to happen. Workload pressure, fatigue, and the gap between the written procedure and the actual practice not least trust in leadership.

Outcomes are easy to count. Conditions are harder to see, and that is exactly why they matter more. If your personnel are working around a badly written procedure every single shift, that will not appear on your dashboard. It will stay invisible until the day the workaround fails.

The question worth sitting with

So here is the question for 2am, not for the next management review.

What if the numbers you check every morning are perfectly accurate and are still telling you almost nothing about your actual safety risk or exposure?

That is not a criticism of your reporting system. Most safety dashboards were inherited, built for compliance rather than insight, and nobody chose to measure the wrong thing on purpose. It happened gradually, one SPI at a time, until the dashboard looked complete without ever describing what keeps you awake.

What this isn't asking you to do

This is not a call to add more metrics. Adding more numbers to a dashboard that already measures the wrong things just gives you more of the wrong things to look at.

It is worth pausing on a narrower question first, quietly, before the next audit cycle. Not “are our numbers good”, but “do our numbers describe what we actually need to know.”

Some organisations find the honest answer is no. That is uncomfortable, and it is also the beginning of a better safety system, one that measures conditions and not just outcomes, and tells you something true, even at 2am.

Categories: : Just Culture, Safety Management Systems (SMS)