Unified Explainable Gated Fusion for Brain Tumor Diagnosis: A Theoretical Framework Extending Cross-Gated Multi-Path Attention Fusion with Multi-Modal Fusion, Segmentation, and Automated Reporting
Article Sidebar
Main Article Content
Abstract
Automated diagnosis of brain tumors from medical images has reached high classification accuracy, but the resulting models rarely localize the lesion, usually accept a single imaging modality, and explain their decisions only post hoc. This paper proposes a theoretical framework that builds on a recently published hybrid of residual convolutional networks and vision transformers joined by cross-gated multi-path attention fusion and trained with a gate-consistency loss. To move from a classifier toward an explainable diagnostic decision-support framework, the model is extended with three components: (a) a masked modality gate that fuses magnetic resonance, computed tomography and positron emission tomography features and remains well defined when some modalities are missing; (b) a shared segmentation decoder; and (c) a traceable explanation and reporting layer. A mathematical formulation is given together with a combined loss in which segmentation is partially supervised, so that samples without masks contribute only to the classification terms. Four properties are proved under stated assumptions: (1) the fusion is a convex combination of the available modality features and is consistent with every availability pattern; (2) branch disagreement is bounded by the gate-consistency penalty; (3) with a linear head, each class score decomposes exactly into location, modality and branch contributions, an algebraic accounting rather than a causal attribution; and (4) integrated-gradient attributions are complete across modalities. An agreement score between the explanation map and the predicted tumor mask, defined for every class including no tumor, flags cases for clinician review. A first-order cost analysis indicates that computation grows linearly with the number of available modalities. No experiments are reported and all claims are theoretical; testable hypotheses and a proof-of-concept protocol on a public dataset with tumor masks are specified for future empirical work.
Article Details
Section
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
All articles published in VTIP-JMRR are made freely available under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Under this license, others may freely share, distribute, and adapt the published work, provided appropriate credit is given to the original authors.
By signing the copyright form, the author(s) warrant and represent that:
✓ The manuscript is original and has not been published previously in any journal, conference, or book (in whole or in part).
✓ The manuscript is not currently under consideration for publication elsewhere.
✓ All authors have read and approved the final manuscript and have agreed to this copyright transfer.
✓ The manuscript does not infringe upon any existing copyright, trademark, patent, or other intellectual property right of any third party.
✓ All data, figures, and tables presented are accurate, authentic, and have not been fabricated or falsified.
✓ Proper permissions have been obtained for any third-party material (images, figures, tables, text) included in the manuscript.
✓ The work complies with all applicable ethical guidelines, including those related to human and animal research.
✓ All sources of funding and conflicts of interest have been disclosed in the manuscript.
How to Cite
References
[1] Muhammad, A., Jin, Q., Musaddiq, M. H., Mir, M. S., Alkanan, M., & Gulzar, Y. (2025, November). Explainable AI for brain tumor classification using cross-gated multi-path attention fusion and gate-consistency loss. IEEE Access, 13, 192731–192746. https://doi.org/10.1109/ACCESS.2025.3629532
[2] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778). https://doi.org/10.1109/CVPR.2016.90
[3] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16×16 words: Transformers for image recognition at scale. In Proceedings of the International Conference on Learning Representations. https://arxiv.org/abs/2010.11929
[4] Khaniki, M. A. L., Mirzaeibonehkhater, M., Manthouri, M., & Hasani, E. (2025, June). Vision transformer with feature calibration and selective cross-attention for brain tumor classification. Iran Journal of Computer Science, 8(2), 335–347. https://doi.org/10.1007/s42044-024-00220-w
[5] Zhou, Z., Yu, X., Huang, W., Qiu, S., Humayoo, M., Li, H., & Shi, M. (2025, December). Comprehensive exploiting local and global features for brain tumor segmentation: A gated dual-branch hybrid attention mechanism. Biomedical Signal Processing and Control, 110, Article 108285. https://doi.org/10.1016/j.bspc.2025.108285
[6] Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017, October). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 618–626). https://doi.org/10.1109/ICCV.2017.74
[7] Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning (Vol. 70, pp. 3319–3328). PMLR.
[8] Akgündoğdu, A., & Çelikbaş, Ş. (2025, October). Explainable deep learning framework for brain tumor detection: Integrating LIME, Grad-CAM, and SHAP for enhanced accuracy. Medical Engineering & Physics, 144, Article 104405. https://doi.org/10.1016/j.medengphy.2025.104405
[9] Saeed, T., Khan, M. A., Hamza, A., Shabaz, M., Khan, W. Z., Alhayan, F., Jamel, L., & Baili, J. (2024, October). Neuro-XAI: Explainable deep learning framework based on DeepLabV3+ and Bayesian optimization for segmentation and classification of brain tumor in MRI scans. Journal of Neuroscience Methods, 410, Article 110247. https://doi.org/10.1016/j.jneumeth.2024.110247
[10] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (pp. 234–241). Springer. https://doi.org/10.1007/978-3-319-24574-4_28
[11] Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., Glocker, B., & Rueckert, D. (2018). Attention U-Net: Learning where to look for the pancreas. arXiv. https://arxiv.org/abs/1804.03999
[12] Jia, Q., & Shu, H. (2022). BiTr-UNet: A CNN-transformer combined network for MRI brain tumor segmentation. In Proceedings of the BrainLesion Workshop (BrainLes) (pp. 3–14). https://doi.org/10.1007/978-3-031-09002-8_1
[13] Huang, Y., Chen, L., & Zhou, C. (2025). Multi-modal brain tumor segmentation via 3D multi-scale self-attention and cross-attention. arXiv. https://arxiv.org/abs/2504.09088
[14] Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., … Van Leemput, K. (2015, October). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10), 1993–2024. https://doi.org/10.1109/TMI.2014.2377694
[15] Milletari, F., Navab, N., & Ahmadi, S.-A. (2016). V-Net: Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the Fourth International Conference on 3D Vision (pp. 565–571). https://doi.org/10.1109/3DV.2016.79
[16] Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd ed.). Wiley-Interscience.
[17] Bray, F., Laversanne, M., Sung, H., Ferlay, J., Siegel, R. L., Soerjomataram, I., & Jemal, A. (2024). Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians, 74(3), 229–263. https://doi.org/10.3322/caac.21834
[18] Havaei, M., Guizard, N., Chapados, N., & Bengio, Y. (2016). HeMIS: Hetero-modal image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016 (LNCS 9901, pp. 469–477). Springer. https://doi.org/10.1007/978-3-319-46723-8_54
[19] Ding, Y., Yu, X., & Yang, Y. (2021). RFNet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 3975–3984). https://doi.org/10.1109/ICCV48922.2021.00394
[20] Zhang, Y., He, N., Yang, J., Li, Y., Wei, D., Huang, Y., Zhang, Y., He, Z., & Zheng, Y. (2022). mmFormer: Multimodal medical transformer for incomplete multimodal learning of brain tumor segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2022 (LNCS 13435, pp. 107–117). Springer. https://doi.org/10.1007/978-3-031-16443-9_11
[21] Jing, B., Xie, P., & Xing, E. (2018). On the automatic generation of medical imaging reports. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (pp. 2577–2586). https://doi.org/10.18653/v1/P18-1240
[22] Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020). Generating radiology reports via memory-driven transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (pp. 1439–1449). https://doi.org/10.18653/v1/2020.emnlp-main.112
[23] Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018). Sanity checks for saliency maps. In Advances in Neural Information Processing Systems 31 (pp. 9505–9515).
[24] Saporta, A., Gui, X., Agrawal, A., Pareek, A., Truong, S. Q. H., Nguyen, C. D. T., Ngo, V. D., Seekins, J., Blankenberg, F. G., Ng, A. Y., Lungren, M. P., & Rajpurkar, P. (2022). Benchmarking saliency methods for chest X-ray interpretation. Nature Machine Intelligence, 4(10), 867–878. https://doi.org/10.1038/s42256-022-00536-x
[25] Cheng, J., Huang, W., Cao, S., Yang, R., Yang, W., Yun, Z., Wang, Z., & Feng, Q. (2015). Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLoS ONE, 10(10), e0140157. https://doi.org/10.1371/journal.pone.0140157