Overpower assumptions with the right kind of evidence

Brian Hughes, Vice President

 

It’s no fun to show up to lead a root cause analysis when the only supporting evidence participants bring is their assumptions.  This, unfortunately, has happened to me more than once.  As a consultant, I’ve learned the hard way to make sure participants have a clear idea what we will need in order to facilitate the investigation.  People need to know what is meant by the word “evidence,” and have a very clear idea of what it means when they are asked to collect evidence to bring to the analysis.  Hopefully this simple overview will provide you with a basic understanding of evidence, evidence types, and how it is used in an investigation.

Adverse events are labeled as “adverse” because they have a negative impact on the goals and objectives of the organization.  Adverse events range in severity level.  Some are extremely – and obviously – impactful.  Others may have a smaller impact, or none at all.  Not all adverse events will be severe enough to warrant a root cause analysis.  But when a formal RCA is indicated, the first step in the process is to gather and manage the data that will be used in the investigation.

In an incident investigation, the data gathered is used as evidence.  Evidence is used to support the investigation.  For instance, if the financial impact of the incident is determined to be USD 1 million, what supports this conclusion?  What is the supporting documentation for this number?  If the team concludes that a cause of the incident is a pressure relief valve that was manually disabled, what is the evidence supporting this?  By seeking up front to find as much potential evidence as possible, we help reduce the risk that the team will come to inaccurate conclusions during the analysis.

When an investigation first begins, the team may not know exactly what will be useful.  Invariably, the team is likely to gather more data than will be needed – and this is okay.  It’s helpful to think about the entire investigation as a process – not unlike mining for gold.  In gold mining, ore is extracted from the ground and goes through a process of refinement until the purity reaches a specified level.  Needless to say, the amount of pure gold that results from the process is much smaller than the amount of ore dug from the ground.  It’s the same in an incident investigation, although not as extreme.  Some of what you gather initially will be very useful, and may even lead to broad systemic opportunities for improvement.  But some won’t.  It’s not easy to predict value at the beginning of an investigation.

Evidence comes in different forms.  Here is a list (in no particular order) containing some of the more prominent sources:

Recorded Evidence:  Recorded evidence is just like it sounds – something that was recorded on some kind of data storage device.  It can take the form of data, audio, and/or video files.  The quality of recorded evidence is generally considered to be high – particularly if the recording device is protected and not easy to tamper with.  A good example of recorded evidence is the picture of the foam strike to the leading edge of the wing of the space shuttle Columbia.  This piece of evidence was discovered upon examining the recorded video footage shot during the launch.

Documentation:  Documents come in many forms, such as written procedures, training records, emails, visitor logs, work orders, purchase orders, etc.  Documentation is a very important source of evidence - even though it is often time consuming to sort through everything collected.

Measurements:  Measurements result from actual physical comparison between something observed and some standard.  When you use a ruler, this is what you are doing – comparing the length of some thing to the standard provided by the ruler.  Measurements come in many forms and as long as the standard being used has been calibrated, they are considered to be sources of high quality evidence.

Expert Opinions:  Calling in the experts is always a good idea, but relying on expert opinions alone can be risky.  Experts can paint fascinating and detailed pictures of the intricacies of a system.  But they can also present opinions as fact.  This can be a risky tendency, even in a fairly simple system.  It is best to corroborate expert opinions with other sources of evidence.

Physical Observation:  Getting statements from people who observed the incident is a must.  They were there – they experienced the event in real time.  But it is not a good idea to rely on these statements alone.  When trying to recall the event in perfect detail, witnesses may not be able to do so accurately, or they may tell a different story than what actually happened.  Memories can be faulty.  And sometimes people act to shape their story in what they think is the most appealing way.  Statements from witnesses, while vital, should be corroborated with other sources.


Post new comment

The content of this field is kept private and will not be shown publicly.
Mollom CAPTCHA (play audio CAPTCHA)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.