Citation

Yildiz AM, Barua PD, Dogan S, Baygin M, Tuncer T, Ooi CP, Fujita H, Rajendra Acharya U. Expert Syst. Appl. 2023; 224: e120031.

Copyright

(Copyright © 2023, Elsevier Publishing)

DOI

10.1016/j.eswa.2023.120031

PMID

unavailable

Abstract

Physical violence detection using multimedia data is crucial for public safety and security. This is an important research area in information security and digital forensics. Research in video-based violence detection (VVD) has grown steadily in recent years with rapid increase in video surveillance systems worldwide. Verbal aggression detection technologies, on the other hand, are still limited due to the popularity of computer vision models. Thus, researchers have preferred to use computer vision models to detect violence using videos. We have presented a new automatic audio violence detection (AVD) model to fill this gap. Our AVD model is handcrafted and its details are as follows. This work collected a new audio dataset on verbal aggression from YouTube. A novel handcrafted model was proposed using multilevel feature extraction, feature selection, classification, and majority voting phases. A new local feature extraction function based on the binary tree was used to generate features from audio signals. We call this function tree pattern-23 (TreePat23), where 23 represents the number of wavelet bands/audio signals inputs. Wavelet bands were generated using tunable Q wavelet transform (TQWT) before being applied to our TreePat23 for feature extraction. The iterative neighborhood component analysis (INCA) and Chi2 were used to select the features. The selected features were classified using k nearest neighbors (kNN) and support vector machine (SVM) followed by iterative majority voting (IMV) method. The best-predicted vector was obtained by using a greedy algorithm. Finally, a new validation technique called leave one record out (LORO) cross-validation (CV) was used to validate the results. Our proposed TreePat23 model has attained classification accuracy of 89.68% and 89.75% with kNN and SVM, respectively. Our developed system has generated 14 results for each classifier and automatically selected the best result. Hence this model is a self-organized audio classification model which yielded over 89% classification accuracy for both classifiers using LORO CV strategy.

Language: en

Keywords

Audio forensics; Audio violence detection; Feature extraction; Iterative feature selection; Signal processing; Tree pattern