Citation

Robinson SD. Safety Sci. 2019; 116: 275-286.

Copyright

(Copyright © 2019, Elsevier Publishing)

DOI

10.1016/j.ssci.2019.03.014

PMID

unavailable

Abstract

The aviation safety reporting system database has seen many applications of topic modeling and natural language processing. Its size, metadata, and narratives have made it ideal for demonstrating new models and approaches. However, less often seen is how these methods are applied by subject matter experts and subsequently interpreted. Within this article, latent Dirichlet allocation is applied to a fourteen year sample of the aviation safety reporting system database, filtered for commercial airline operations. The topics terms generated were presented to three subject matter experts, along with weighted topic narratives, and mean usage of each topic by month. A high level of inter-rater agreement was demonstrated for the conceptual themes that emerged from the selected topics. The subject matter experts were able to identify the reporter's qualification (air traffic controllers, etc.) as well as some of the environmental, regulatory, and industry factors consistent with observed temporal trends of topic usage. Through this examination, additional suggestions are made for the process of interpretation of topic modeled themes, expanding on the work of DiMaggio, Nag, and Blei. Post facto, the study identified the increase in reported safety concerns facing flight crews following the introduction of the runway awareness and advisory system into the cockpit in 2009. Additionally, the results precipitated a change to the way the subject matter experts interpreted the narratives and their implications to industry safety. This work provides a practical validation of the use of natural language processing in sensemaking and pinpointing trends necessary for prioritizing safety activities.

Language: en

Keywords

Aviation safety; Natural language processing; Sensemaking; Temporal trends; Topic modeling