A ROBUST MACHINE LEARNING FRAMEWORK FOR ANOMALY-BASED INTRUSION DETECTION IN SOFTWARE-DEFINED NETWORKS

Abstract

Software Defined Networking (SDN) enables centralized network management, enhanced programmability, and improved flexibility by decoupling the control plane from the data plane. However, this architecture introduces security vulnerabilities that challenge conventional signature-based Intrusion Detection Systems (IDS), which are often ineffective against evolving or novel cyber threats. This study investigates the efficiency of ensemble machine learning and deep learning models for anomaly-based IDS within SDN environments. The CIC-IDS-2017 benchmark dataset was employed for evaluation due to its realistic traffic patterns and inclusion of modern attack scenarios. A multi-stage preprocessing pipeline was applied, including Min-Max normalization, label encoding and the Synthetic Minority Over-sampling Technique (SMOTE) to address severe class imbalance in network traffic data. Several machine learning techniques, including XGBoost, Random Forest (RF), and Support Vector Machine (SVM), were examined alongside a lightweight custom-designed Artificial Neural Network (ANN). The suggested ANN contains five layers, an input layer, three hidden layers, and one output layer, designed to capture complex nonlinear patterns in network traffic. Experimental results show that the ANN achieved superior performance with a peak accuracy of 99.80%, while among machine learning models; RF outperformed all others with an accuracy of 99.05%. These findings indicate that deep learning-based approaches are highly effective in detecting diverse intrusion types, capturing complex traffic patterns, and maintaining low false negative and false positive rates, highlighting their suitability for SDN security.