REAL-TIME SOUND EVENT LOCALIZATION AND DETECTION USING IOT RASPBERRY PI DEVICES BASED ON SINGLE-STAGE CRNN
Abstract
Sound event localization and detection (SELD) plays a vital role in understanding the environment. Recently, the SELD problem has received increasing interest from the research community. The state-of-the-art models for the DCASE 2020 for the SELD task have achieved good performance in terms of accuracy, but these models are generally based on multistage deep neural networks, which require substantial computational power and memory, making them unsuitable for deployment on low-cost Internet of Things (IoT) edge devices. In this paper, we propose a single-stage convolutional recurrent neural network (SS-CRNN) designed for real-time implementation of SELD on resource-constrained devices like Raspberry Pi. We also conducted a comprehensive analysis of different feature representations in terms of both accuracy and computational efficiency. Our results demonstrate that the SS-CRNN outperforms other models on the DCASE 2020 SELD dataset in terms of real-time factor (RTF), with only a slight trade-off: a 1% reduction in frame recall performance and a 6-degree decrease in localization accuracy compared to state-of-the-art methods. Additionally, we employ SpecMix augmentation to further enhance the model’s performance, which helps to boost our model’s performance during training.
