(Universitas Muhammadiyah Semarang, Indonesia)(2) Norshuhani Zamin
(Universiti Tenaga Nasional (UNITEN), Malaysia)(3) Muhammad Sam’an
(Universitas Muhammadiyah Semarang, Indonesia)*corresponding author
AbstractPneumonia detection through medical imaging, especially using CT scans or X-rays, presents notable challenges due to the subtle and often unclear signs of the disease. This paper introduces a novel neural network model, the Compact Convolutional Transformer (CCT), designed to address these challenges by optimizing detection accuracy. The CCT model incorporates configuration dropout in its convolutional layers to enhance both robustness and precision.Experiments conducted on a dataset of 5,856 chest X-ray images from pediatric patients aged one to five years demonstrated the model's effectiveness, achieving a remarkable 97% accuracy, 97% recall, 98% precision, and an F1-score of 98%. When compared to state-of-the-art models like DarkNet-53 and VGG-19 + GradCAM, which achieved F1-scores of 97.3% and 95.61% respectively, the CCT model consistently matched or outperformed them, particularly when dealing with smaller and more complex datasets. Even models such as CNN + Bayesian Network, which used larger datasets, only reached an F1-score of 96.3%.These results underscore the superior efficiency and accuracy of the CCT model, highlighting its potential for broader applications in medical diagnostics and image analysis, especially in pneumonia detection.
KeywordsPneumonia detection, Compact convolution transformer ,CNN, CCT ,X-Ray Image
|
DOIhttps://doi.org/10.26555/ijain.v12i2.1805 |
Article metricsAbstract views : 245 | PDF views : 45 |
Cite |
Full Text Download
|
References
[1] H. Campbell et al., “Measuring Coverage in MNCH: Challenges in Monitoring the Proportion of Young Children with Pneumonia Who Receive Antibiotic Treatment,” PLoS Med., vol. 10, no. 5, p. e1001421, May 2013, doi: 10.1371/journal.pmed.1001421.
[2] R. Jain, P. Nagrath, G. Kataria, V. Sirish Kaushik, and D. Jude Hemanth, “Pneumonia detection in chest X-ray images using convolutional neural networks and transfer learning,” Measurement, vol. 165, no. December, p. 108046, Dec. 2020, doi: 10.1016/j.measurement.2020.108046.
[3] S. Su, D. Yuan, Y. Wang, and M. Ding, “Fine Grained Feature Extraction Model of Riot-related Images Based on YOLOv5,” Comput. Syst. Sci. Eng., vol. 45, no. 1, pp. 85–97, Aug. 2023, doi: 10.32604/csse.2023.030849.
[4] R. Karthik, R. Menaka, and H. M., “Learning distinctive filters for COVID-19 detection from chest X-ray using shuffled residual CNN,” Appl. Soft Comput., vol. 99, no. February, p. 106744, Feb. 2021, doi: 10.1016/j.asoc.2020.106744.
[5] H. Quan, X. Xu, T. Zheng, Z. Li, M. Zhao, and X. Cui, “DenseCapsNet: Detection of COVID-19 from X-ray images using a capsule neural network,” Comput. Biol. Med., vol. 133, no. June, p. 104399, Jun. 2021, doi: 10.1016/j.compbiomed.2021.104399.
[6] A. Alhudhaif, K. Polat, and O. Karaman, “Determination of COVID-19 pneumonia based on generalized convolutional neural network model from chest X-ray images,” Expert Syst. Appl., vol. 180, no. October, p. 115141, Oct. 2021, doi: 10.1016/j.eswa.2021.115141.
[7] A. K. Jaiswal, P. Tiwari, S. Kumar, D. Gupta, A. Khanna, and J. J. P. C. Rodrigues, “Identifying pneumonia in chest X-rays: A deep learning approach,” Measurement, vol. 145, no. October, pp. 511–518, Oct. 2019, doi: 10.1016/j.measurement.2019.05.076.
[8] L. Brunese, F. Mercaldo, A. Reginelli, and A. Santone, “Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-rays,” Comput. Methods Programs Biomed., vol. 196, no. November, p. 105608, Nov. 2020, doi: 10.1016/j.cmpb.2020.105608.
[9] T. Mahmud, M. A. Rahman, and S. A. Fattah, “CovXNet: A multi-dilation convolutional neural network for automatic COVID-19 and other pneumonia detection from chest X-ray images with transferable multi-receptive feature optimization,” Comput. Biol. Med., vol. 122, no. July, p. 103869, Jul. 2020, doi: 10.1016/j.compbiomed.2020.103869.
[10] C. Ouchicha, O. Ammor, and M. Meknassi, “CVDNet: A novel deep learning architecture for detection of coronavirus (Covid-19) from chest x-ray images,” Chaos, Solitons & Fractals, vol. 140, no. November, p. 110245, Nov. 2020, doi: 10.1016/j.chaos.2020.110245.
[11] J. Wang et al., “Prior-Attention Residual Learning for More Discriminative COVID-19 Screening in CT Images,” IEEE Trans. Med. Imaging, vol. 39, no. 8, pp. 2572–2583, Aug. 2020, doi: 10.1109/TMI.2020.2994908.
[12] H. Yang, K. Zhu, D. Huang, H. Li, Y. Wang, and L. Chen, “Intensity enhancement via GAN for multimodal face expression recognition,” Neurocomputing, vol. 454, no. September, pp. 124–134, Sep. 2021, doi: 10.1016/j.neucom.2021.05.022.
[13] Y. Wang et al., “A systematic review on affective computing: emotion models, databases, and recent advances,” Inf. Fusion, vol. 83–84, no. July, pp. 19–52, Jul. 2022, doi: 10.1016/j.inffus.2022.03.009.
[14] S.-H. Wang, X. Zhang, and Y.-D. Zhang, “DSSAE: Deep Stacked Sparse Autoencoder Analytical Model for COVID-19 Diagnosis by Fractional Fourier Entropy,” ACM Trans. Manag. Inf. Syst., vol. 13, no. 1, pp. 1–20, Mar. 2022, doi: 10.1145/3451357.
[15] S. Rajaraman, J. Siegelman, P. O. Alderson, L. S. Folio, L. R. Folio, and S. K. Antani, “Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays,” IEEE Access, vol. 8, pp. 115041–115050, 2020, doi: 10.1109/ACCESS.2020.3003810.
[16] P. Szepesi and L. Szilágyi, “Detection of pneumonia using convolutional neural networks and deep learning,” Biocybern. Biomed. Eng., vol. 42, no. 3, pp. 1012–1022, Jul. 2022, doi: 10.1016/j.bbe.2022.08.001.
[17] K. Tyagi, G. Pathak, R. Nijhawan, and A. Mittal, “Detecting Pneumonia using Vision Transformer and comparing with other techniques,” in 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA), IEEE, Dec. 2021, pp. 12–16. doi: 10.1109/ICECA52323.2021.9676146.
[18] S. Gupta, A. Rodrigues, P. Joshi, and J. George, “An effective Approach for Pneumonia Detection using Convolution Vision Transformer,” in 2022 International Conference on Trends in Quantum Computing and Emerging Business Technologies (TQCEBT), IEEE, Oct. 2022, pp. 1–6. doi: 10.1109/TQCEBT54229.2022.10041662.
[19] P. N. Ha, A. Doucet, and G. S. Tran, “Vision Transformer for Pneumonia Classification in X-ray Images,” in Proceedings of the 2023 8th International Conference on Intelligent Information Technology, New York, NY, USA: ACM, Feb. 2023, pp. 185–192. doi: 10.1145/3591569.3591602.
[20] L. Zhang and Y. Wen, “A transformer-based framework for automatic COVID19 diagnosis in chest CTs,” in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), IEEE, Oct. 2021, pp. 513–518. doi: 10.1109/ICCVW54120.2021.00063.
[21] D. Kermany, K. Zhang, and M. Goldbaum, “Large Dataset of Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images,” vol. 3, 2018, Accessed: May 31, 2026. [Online]. Available: https://data.mendeley.com/datasets/rscbjbr9sj/3.
[22] J. Chai, H. Zeng, A. Li, and E. W. T. Ngai, “Deep learning in computer vision: A critical review of emerging techniques and application scenarios,” Mach. Learn. with Appl., vol. 6, no. December, p. 100134, Dec. 2021, doi: 10.1016/j.mlwa.2021.100134.
[23] S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in 2017 International Conference on Engineering and Technology (ICET), IEEE, Aug. 2017, pp. 1–6. doi: 10.1109/ICEngTechnol.2017.8308186.
[24] A. Saxena, “An Introduction to Convolutional Neural Networks,” Int. J. Res. Appl. Sci. Eng. Technol., vol. 10, no. 12, pp. 943–947, Dec. 2022, doi: 10.22214/ijraset.2022.47789.
[25] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, May 2017, doi: 10.1145/3065386.
[26] R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, “Convolutional neural networks: an overview and application in radiology,” Insights Imaging, vol. 9, no. 4, pp. 611–629, Aug. 2018, doi: 10.1007/s13244-018-0639-9.
[27] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, and F. E. Alsaadi, “A survey of deep neural network architectures and their applications,” Neurocomputing, vol. 234, no. April, pp. 11–26, Apr. 2017, doi: 10.1016/j.neucom.2016.12.038.
[28] I. Masi, Y. Wu, T. Hassner, and P. Natarajan, “Deep Face Recognition: A Survey,” in 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), IEEE, Oct. 2018, pp. 471–478. doi: 10.1109/SIBGRAPI.2018.00067.
[29] T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 38–45. doi: 10.18653/v1/2020.emnlp-demos.6.
[30] S. Edunov, M. Ott, M. Auli, and D. Grangier, “Understanding Back-Translation at Scale,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2018, pp. 489–500. doi: 10.18653/v1/D18-1045.
[31] Z. Abbasiantaeb and S. Momtazi, “Text‐based question answering from information retrieval and deep neural network perspectives: A survey,” WIREs Data Min. Knowl. Discov., vol. 11, no. 6, p. e1412, Nov. 2021, doi: 10.1002/widm.1412.
[32] O. Habimana, Y. Li, R. Li, X. Gu, and G. Yu, “Sentiment analysis using deep learning approaches: an overview,” Sci. China Inf. Sci., vol. 63, no. 1, p. 111102, Jan. 2020, doi: 10.1007/s11432-018-9941-6.
[33] S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in Vision: A Survey,” ACM Comput. Surv., vol. 54, no. 10s, pp. 1–41, Jan. 2022, doi: 10.1145/3505244.
[34] A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in ICLR 2021 - 9th International Conference on Learning Representations, International Conference on Learning Representations, ICLR, Jun. 2021, p. 22. doi: 10.48550/arXiv.2010.11929.
[35] A. Vaswani et al., “Attention is All you Need,” in Advances in Neural Information Processing Systems, 2017, p. 11. doi: 10.48550/arXiv.1706.03762.
[36] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North, Stroudsburg, PA, USA: Association for Computational Linguistics, 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.
[37] A. K. Dash and P. Mohapatra, “A Fine-tuned deep convolutional neural network for chest radiography image classification on COVID-19 cases,” Multimed. Tools Appl., vol. 81, no. 1, pp. 1055–1075, Jan. 2022, doi: 10.1007/s11042-021-11388-9.
[38] H. Panwar, P. K. Gupta, M. K. Siddiqui, R. Morales-Menendez, P. Bhardwaj, and V. Singh, “A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images,” Chaos, Solitons & Fractals, vol. 140, no. November, p. 110190, Nov. 2020, doi: 10.1016/j.chaos.2020.110190.
[39] M. E. H. Chowdhury et al., “Can AI Help in Screening Viral and COVID-19 Pneumonia?,” IEEE Access, vol. 8, pp. 132665–132676, 2020, doi: 10.1109/ACCESS.2020.3010287.
[40] H. Ren et al., “Interpretable Pneumonia Detection by Combining Deep Learning and Explainable Models With Multisource Data,” IEEE Access, vol. 9, pp. 95872–95883, 2021, doi: 10.1109/ACCESS.2021.3090215.
[41] J. D. Arias-Londono, J. A. Gomez-Garcia, L. Moro-Velazquez, and J. I. Godino-Llorente, “Artificial Intelligence Applied to Chest X-Ray Images for the Automatic Detection of COVID-19. A Thoughtful Evaluation Approach,” IEEE Access, vol. 8, pp. 226811–226827, 2020, doi: 10.1109/ACCESS.2020.3044858.
[42] S. Sakib, T. Tazrin, M. M. Fouda, Z. M. Fadlullah, and M. Guizani, “DL-CRC: Deep Learning-Based Chest Radiograph Classification for COVID-19 Detection: A Novel Approach,” IEEE Access, vol. 8, pp. 171575–171589, 2020, doi: 10.1109/ACCESS.2020.3025010.
[43] T. Ozturk, M. Talo, E. A. Yildirim, U. B. Baloglu, O. Yildirim, and U. Rajendra Acharya, “Automated detection of COVID-19 cases using deep neural networks with X-ray images,” Comput. Biol. Med., vol. 121, no. June, p. 103792, Jun. 2020, doi: 10.1016/j.compbiomed.2020.103792.
[44] A. K. Das, S. Ghosh, S. Thunder, R. Dutta, S. Agarwal, and A. Chakrabarti, “Automatic COVID-19 detection from X-ray images using ensemble learning with convolutional neural network,” Pattern Anal. Appl., vol. 24, no. 3, pp. 1111–1124, Aug. 2021, doi: 10.1007/s10044-021-00970-4.
[45] H. Munusamy, K. J. Muthukumar, S. Gnanaprakasam, T. R. Shanmugakani, and A. Sekar, “FractalCovNet architecture for COVID-19 Chest X-ray image Classification and CT-scan image Segmentation,” Biocybern. Biomed. Eng., vol. 41, no. 3, pp. 1025–1038, Jul. 2021, doi: 10.1016/j.bbe.2021.06.011.
[46] R. C. Joshi et al., “A deep learning-based COVID-19 automatic diagnostic framework using chest X-ray images,” Biocybern. Biomed. Eng., vol. 41, no. 1, pp. 239–254, Jan. 2021, doi: 10.1016/j.bbe.2021.01.002.
[47] S. Singh and B. K. Tripathi, “Pneumonia classification using quaternion deep learning,” Multimed. Tools Appl., vol. 81, no. 2, pp. 1743–1764, Jan. 2022, doi: 10.1007/s11042-021-11409-7.
[48] M. Gour and S. Jain, “Automated COVID-19 detection from X-ray and CT images with stacked ensemble convolutional neural network,” Biocybern. Biomed. Eng., vol. 42, no. 1, pp. 27–41, Jan. 2022, doi: 10.1016/j.bbe.2021.12.001.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0