Citation

Minotra D, McNeese MD. Cogn. Technol. Work 2017; 19(1): 161-177.

Copyright

(Copyright © 2017, Holtzbrinck Springer Nature Publishing Group)

DOI

10.1007/s10111-017-0402-x

PMID

unavailable

Abstract

Task performance in uncertain and complex environments is dependent on human effectiveness in dynamic prioritisation and attention allocation. Non-routine critical events place a high demand on cognitive resources in operators, and they require operators to dynamically re-prioritise sub-tasks calling upon the need for predictive aids. Predictive aids provide real-time prediction of system variables, and they can potentially facilitate the detection of non-routine critical events and dynamic re-prioritisation during such events. However, prior research has reported mixed findings in their effectiveness, and no prior research has been done to test the effectiveness of predictive aids during non-routine critical events that require operators to dynamically prioritise tasks. Our experimental study with 77 participants examined the effect of a predictive aid on prioritisation of non-routine critical events in a cyber security event monitoring task. The predictive aid resulted in decrements in sustained attention as seen in delayed detection of some non-routine critical events and errors in prioritising non-routine critical events. This effect is a result of miscalibration in the importance of looking ahead with the predictive aid. This miscalibration was possibly associated with the look-ahead time, complexity of the predictive aid, and the lack of automation transparency (i.e. mechanism underlying prediction) together affecting its perceived usefulness. The experimental results have the following implications on the design and testing of predictive aids: (1) in a task requiring dynamic prioritisation and detection of non-routine critical events, predictive aids with insufficient look-ahead times can result in decrements in sustained attention; (2) predictive aids should be evaluated with subjective measures of perceived usefulness and workload; (3) predictive aids should be designed to have sufficient look-ahead times and transparency of underlying prediction mechanisms as these would affect visual scanning effort and perceived usefulness; and (4) experimental tests of their effectiveness should involve scenarios long enough that would take sustained attention effects and any potential learning effects into consideration.

Language: en