MACHINE LEARNING-BASED THREAT DETECTION IN LOW-RESOURCE URDU TWEETS: COMPARATIVE EVALUATION OF CLASSICAL MODELS

Abstract

The high rate of social media websites development has brought positive opportunities as well as bad ones, such as threats, harassment, and hate speech. Twitter is a highly volatile source of short messaging or tweets, which makes it very difficult to identify threats in real time because of use of informal language, abbreviations as well as user-defined slang. Machine learning-based automated detection systems will be needed to overcome this challenge. This paper provides a comparative study of four classical machine learning classifiers, namely Logistic Regression, Random Forest, Naive Bayes, and Support Vector machine (SVM), in the threat detection process of the tweets. The authors used 3,170 manually labeled tweets (2,891 non-threat and 279 threat) as the dataset of the study. The preprocessing tools such as tokenization, stop-words elimination and TF-IDF vectorization are used to transform textual data into numbers that one can use to classify. Standard metrics used to measure the model include precision, recall, F1-score, and accuracy. The findings show a balanced performance with a high accuracy of 82.69 which shows that SVM is most accurate in both threat and non-threat classes. Logistic Regression and Naive Bayes give competitive results but slightly with less accuracy whereas Random Forest is robust yet sensitive to class imbalance. This discussion indicates that conventional machine learning classifiers are successful in detecting threats to the social media and serves as a reference point in future studies on the use of deep learning models and real-time use cases.