AI-Driven Anomaly Detection Frameworks for Real-Time Payment System Compliance

Authors:
Padma Vayuvegula

Addresses:
Department of Banking, University of the Cumberlands, Williamsburg, Kentucky, United States of America.

Abstract:

Finance is moving at light speed towards Real-Time Payment Systems (RTPS), which, with as great a head start as they have over unimaginable convenience and speed, are beset with humongous compliance risk in the guise of hi-tech fraud and money laundering. The traditional rule-based monitoring architecture cannot keep pace with the speed and volume of such transactions; therefore, there is a historical need for intelligent analytical solutions. The current paper proposes a novel hybrid AI model for RPS anomaly detection. The model utilises the synergy of maximising the power of employing an Isolation Forest algorithm to effectively detect outliers and that of an Autoencoder neural network to learn non-linear, implicit features from transactional data. The data used was a synthetically generated dataset of 432 samples, constructed to include both regular and anomalous transactions. We developed and tested our environment using Python and libraries such as Scikit-learn for Isolation Forest and TensorFlow/Keras for Autoencoder. Our results indicate that the hybrid model presented in this paper achieves higher F1 Scores and AUC-ROC than standalone models, and significantly improves precision and efficiency in identifying fraudulent transactions. This paper presents an effective and scalable solution for banks to enhance their compliance processes and ensure the security of real-time payment systems.

Keywords: Anomaly Detection; Artificial Intelligence; Real-Time Payments; Financial Compliance; Machine Learning; Financial Transactions; Deep Learning; Transactional Data; Precision and Efficiency.

Received on: 03/12/2024, Revised on: 08/02/2025, Accepted on: 22/03/2025, Published on: 05/09/2025

DOI: 10.69888/FTSCL.2025.000430

FMDB Transactions on Sustainable Computer Letters, 2025 Vol. 3 No. 3, Pages: 150-157

  • Views : 73
  • Downloads : 7
Download PDF