Explainable AI-Based Financial Fraud Detection Using Big Data Analytics: A Transparent and Adaptive Machine Learning Framework

Financial fraud is costly, causes significant economic losses and undermines confidence in the digital payment systems, while the best machine learning models for detection tend to be opaque and fail when fraudster activity shifts. In this study, we canvass a clear and adaptive fraud detection framework pairing together an imbalance aware gradient boosting classifier with Shapley-value explanations and drift-aware retraining loop for big data streaming setting. The framework is tested on a 120,000-transaction synthetic payment stream that contains a 2.0% fraud rate and a sudden switch in fraud tactics at week 17; the simulator is provided along with the paper to allow for reproduction of the study. During the pre-drift hold-out period, all four learners (logistic regression, random forest, XGBoost, LightGBM) achieved ROC-AUC scores ranging from 0.933 to 0.940, with XGBoost achieving the highest precision–recall AUC of 0.366. The regime change achieved a post-drift mean PR-AUC of 0.361 compared to the static model (0.263) and the fixed four-week retraining (0.321), and the highest mean F1-score (0.357 vs 0.328 and 0.318). SHAP explanations had a high degree of stability across bootstrap models (mean top-5 Jaccard similarity of 0.78); they had good agreement with permutation importance (Spearman ρ = 0.78) and passed a deletion test were masking the three most important features reduced ROC-AUC by 0.039 compared to 0.022 for random masking. The production monitoring, based on input- and score-distribution monitoring, did not indicate the concept drift, which emphasizes the need for label-feedback monitoring. We talk about the regulatory implications, synthetic evaluation and limitations and give an overview of validation on real transaction data.