COMPUTER VISION TRANSFORMER WITH SPARSE ATTENTION FOR LOW-LATENCY REAL-TIME OBJECT DETECTION

Abstract

Real-time object detection has emerged as a necessity in intelligent surveillance systems, self-driving cars, robotics and edge computing applications that demand both low latency and high object detection accuracy. Traditional CNN-based detectors offer fast inference but usually fail to consider long-distance context, while ViTs provide better features' representation at the cost of increased computational complexity and inference latency. This research work introduces Computer Vision Transformer with Sparse Attention (CVT-SA) that enables real-time object detection through reduction of irrelevant attention calculations without sacrificing detection accuracy. Proposed model was tested on the COCO 2017 benchmark dataset that consists of 118,287 images for training and 5,000 images for testing. Evaluation of the model was done in comparison with YOLOv8, DETR, EfficientFormer and traditional Vision Transformers based on precision, recall, mean Average Precision (mAP), inference latency, throughput, GPU usage, and Frames Per Second (FPS). Based on experimental results, the proposed model of sparse attention yielded a detection accuracy of 95.8%, 93.9% precision, 94.7% recall, and 94.8% mAP while decreasing the latency for inference to 18.6 ms/frame. The computational complexity was reduced by 41.3%, the amount of GPU memory was lowered by 35.4%, and the total processing time was decreased by 38.7% when compared with the standard Vision Transformer. The framework operates at a speed of 53.8 frames per second (FPS), which makes it appropriate for real-time implementation on limited resources devices. Furthermore, based on comparative analysis, better performance is observed for small object detection and crowd scenes recognition with reduced false detection rates. It can be concluded that application of sparse attention mechanism to Vision Transformers allows achieving a good compromise between computational efficiency and detection accuracy.