Comparative Evaluation of Naive Bayes, SVM, KNN, Random Forest, and XGBoost for Malware Classification

Authors:
Hamza Amir Saeed

Addresses:
Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, England, United Kingdom.

Abstract:

Billions of new malware threats are produced each year, which can damage computers and steal data. Because it uses similar templates, aging antivirus software rarely detects new harmful code. This study analyses a naive machine learning technique for distinguishing malware based on fixed properties, such as code instructions (opcodes) and file structure (PE headers), using the Microsoft Malware Classification Challenge dataset, which contains over 10,000 malware samples from 9 families. Our models are tested on five models: Naive Bayes to check basic probability, Support Vector Machine to show clear boundaries, K-Nearest Neighbours to show neighbouring examples, random forest to show group tree votes, and random forest to show step-by-step tree fixes. Data is evenly distributed and balanced to eliminate imbalance. Naive Bayes and XGBoost had 76.59-99.54 percent accuracy and 0.60-0.98 F1 scores on 2,174 test samples. XGBoost and other ensembles, such as Random Forests, are best for unusual families with less than 1% error and balanced classes. Opcode features like mov have the greatest impact (up to 30%), enabling us to reduce data by 80% without losing any genes. This is 3-5 times faster and easier than deep learning baselines. The effort provides a pipeline, testing standards, and AV tool ideas, such as quick cell phone scans. Dynamic checks are being developed to better detect zero-day threats.

Keywords: Machine Learning (ML); Naive Bayes; Support Vector Machine (SVM); K-Nearest Neighbours (KNNs); Random Forest (RF); Malware Classification; Opcode Features.

Received on: 22/04/2025, Revised on: 03/07/2025, Accepted on: 04/09/2025, Published on: 05/06/2026

DOI: 10.69888/FTSIN.2026.000706

FMDB Transactions on Sustainable Intelligent Networks, 2026 Vol. 3 No. 2, Pages: 85-112

  • Views : 67
  • Downloads : 12
Download PDF