Citation

Kirwan B. Safety Sci. 1997; 27(1): 43-75.

Copyright

(Copyright © 1997, Elsevier Publishing)

DOI

unavailable

PMID

unavailable

Abstract

This is the second of two papers reviewing the validity of a set of nine Human Reliability Assessment (HRA) quantification techniques. These techniques play a critical role in assessing risk for large, complex, and potentially hazardous installations such as nuclear power plants. It is therefore important to demonstrate the accuracy and validity of these techniques. This is achieved by validations, where the techniques' predictions are compared against historical data, or some other form of benchmark, to see how accurate the predictions are. The first paper therefore produced a common set of validation criteria, so that comparisons could be made uniformly across different validation studies. The primary criteria are accuracy (a statistically significant predictive relationship) and precision (predictions to be within a factor of ten of the true values). Secondary criteria relate to optimism/pessimism of estimates, and inter-judge/assessor consistency, etc. The first paper also noted, however, that these various studies have differed in their experimental rigour and the quality of data/subjects they have used. Therefore, a set of classifications for determining the relative quality of evidence of different validation studies was developed. This will enable conflicting evidence to be resolved in many cases, as the evidence from certain studies will be given more weight than for other studies. The first part of this second paper summarises twenty-two HRA quantification technique validations, including one previously unpublished comparative validation. The results of these validations are then used to determine which techniques are currently supported by empirical evidence, which techniques have been shown to be invalid, and which techniques have been insufficiently validated. Some of the validation issues raised in the first paper are re-visited following the review of validations, and recommendations are made in terms of formats for future validations. Furthermore, it is concluded that ultimately validations of HRA must address the whole HRA process, including task and error analysis, and not merely the quantification component, and must also begin to focus more on internal validity issues.