Journal Home > Volume 5 , Issue 2

The potential of improving disease detection and treatment planning comes with accurate and fully automatic algorithms for brain tumor segmentation. Glioma, a type of brain tumor, can appear at different locations with different shapes and sizes. Manual segmentation of brain tumor regions is not only time-consuming but also prone to human error, and its performance depends on pathologists’ experience. In this paper, we tackle this problem by applying a fully convolutional neural network SegNet to 3D data sets for four MRI modalities (Flair, T1, T1ce, and T2) for automated segmentation of brain tumor and sub-tumor parts, including necrosis, edema, and enhancing tumor. To further improve tumor segmentation, the four separately trained SegNet models are integrated by post-processing to produce four maximum feature maps by fusing the machine-learned feature maps from the fully convolutional layers of each trained model. The maximum feature maps and the pixel intensity values of the original MRI modalities are combined to encode interesting information into a feature representation. Taking the combined feature as input, a decision tree (DT) is used to classify the MRI voxels into different tumor parts and healthy brain tissue. Evaluating the proposed algorithm on the dataset provided by the Brain Tumor Segmentation 2017 (BraTS 2017) challenge, we achieved F-measure scores of 0.85, 0.81, and 0.79 for whole tumor, tumor core, and enhancing tumor, respectively.

Experimental results demonstrate that using SegNet models with 3D MRI datasets and integrating the four maximum feature maps with pixel intensity values of the original MRI modalities has potential to perform well on brain tumor segmentation.


menu
Abstract
Full text
Outline
About this article

Automated brain tumor segmentation on multi-modal MR image using SegNet

Show Author's information Salma Alqazzaz1,2( )Xianfang Sun3Xin Yang1Len Nokes1
School of Engineering, Cardiff University, Cardiff, CF24 3AA, UK.
Department of Physics, College of Science for Women, Baghdad University, Baghdad, Iraq.
School of Computer Science and Informatics, Cardiff University, Cardiff, CF24 3AA, UK.

Abstract

The potential of improving disease detection and treatment planning comes with accurate and fully automatic algorithms for brain tumor segmentation. Glioma, a type of brain tumor, can appear at different locations with different shapes and sizes. Manual segmentation of brain tumor regions is not only time-consuming but also prone to human error, and its performance depends on pathologists’ experience. In this paper, we tackle this problem by applying a fully convolutional neural network SegNet to 3D data sets for four MRI modalities (Flair, T1, T1ce, and T2) for automated segmentation of brain tumor and sub-tumor parts, including necrosis, edema, and enhancing tumor. To further improve tumor segmentation, the four separately trained SegNet models are integrated by post-processing to produce four maximum feature maps by fusing the machine-learned feature maps from the fully convolutional layers of each trained model. The maximum feature maps and the pixel intensity values of the original MRI modalities are combined to encode interesting information into a feature representation. Taking the combined feature as input, a decision tree (DT) is used to classify the MRI voxels into different tumor parts and healthy brain tissue. Evaluating the proposed algorithm on the dataset provided by the Brain Tumor Segmentation 2017 (BraTS 2017) challenge, we achieved F-measure scores of 0.85, 0.81, and 0.79 for whole tumor, tumor core, and enhancing tumor, respectively.

Experimental results demonstrate that using SegNet models with 3D MRI datasets and integrating the four maximum feature maps with pixel intensity values of the original MRI modalities has potential to perform well on brain tumor segmentation.

Keywords: convolutional neural networks, brain tumor segmentation, multi-modal MRI, fully convolutional networks, decision tree

References(26)

[1]
Louis, D. N.; Perry, A.; Reifenberger, G.; von Deimling, A.; Figarella-Branger, D.; Cavenee, W. K.; Ohgaki, H.; Wiestler, O. D.; Kleihues, P.; Ellison, D. W. The 2016 world health organization classification of tumors of the central nervous system: A summary. Acta Neuropathologica Vol. 131, No. 6, 803-820, 2016.
[2]
Menze, B.; Reyes, M.; van Leemput, K. The multimodal brain tumorimage segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging Vol. 34, No. 10, 1993-2024, 2015.
[3]
Juan-Albarracín, J.; Fuster-Garcia, E.; Manjón, J. V.; Robles, M.; Aparici, F.; Martí-Bonmatí, L.; García-Gómez, J. M. Automated glioblastoma segmentation based on a multiparametric structured unsupervised classification. PLoS ONE Vol. 10, No. 5, e0125143, 2015.
[4]
Bauer, S.; Wiest, R.; Nolte, L. P.; Reyes, M. A survey of MRI-based medical image analysis for brain tumor studies. Physics in Medicine and Biology Vol. 58, No. 13, R97-R129, 2013.
[5]
Havaei, M.; Davy, A.; Warde-Farley, D.; Biard, A.; Courville, A.; Bengio, Y.; Pal, C.; Jodoin, P. M.; Larochelle, H. Brain tumor segmentation with deep neural networks. Medical Image Analysis Vol. 35, 18-31, 2017.
[6]
Bakas, S.; Zeng, K.; Sotiras, A.; Rathore, S.; Akbari, H.; Gaonkar, B.; Rozycki, M.; Pati, S.; Davatzikos, C.GLISTRboost: Combining multimodal MRI segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 9556. Crimi, A.; Menze, B.; Maier, O.; Reyes, M.; Handels, H. Eds. Springer Cham, 144-155, 2016.
DOI
[7]
Pereira, S.; Pinto, A.; Alves, V.; Silva, C. A. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging Vol. 35, No. 5, 1240-1251, 2016.
[8]
Shelhamer, E.; Long, J.; Darrell, T. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 4, 640-651, 2017.
[9]
Ronneberger, O.; Fischer, P.; Brox, T.U-Net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Navab, N.; Hornegger, J.; Wells, W.; Frangi, A. Eds. Springer Cham, 234-241, 2015.
DOI
[10]
Shah, S. A.; Chauhan, N. Techniques for detection and analysis of tumours from brain MRI images: A review. Journal of Biomedical Engineering and Medical Imaging Vol. 3, No. 1, 9-20, 2016.
[11]
Gooya, A.; Pohl, K. M.; Bilello, M.; Biros, G.; Davatzikos, C.Joint segmentation and deformable registration of brain scans guided by a tumor growth model. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2011. Lecture Notes in Computer Science, Vol. 6892. Fichtinger, G.; Martel, A.; Peters, T. Eds. Springer Berlin Heidelberg, 532-540, 2011.
DOI
[12]
Zikic, D.; Glocker, B.; Konukoglu, E.; Criminisi, A.; Demiralp, C.; Shotton, J.; Thomas, O. M.; Das, T.; Jena, R.; Price, S. J.Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. Lecture Notes in Computer Science, Vol. 7512. Ayache, N.; Delingette, H.; Golland, P.; Mori, K. Eds. Springer Berlin Heidelberg, 369-376, 2012.
DOI
[13]
Tustison, N. J.; Shrinidhi, K. L.; Wintermark, M.; Durst, C. R.; Kandel, B. M.; Gee, J. C.; Grossman, M. C.; Avants, B. B. Optimal symmetric multimodal templates and concatenated random forests for supervised brain tumor segmentation (simplified) with ANTsR. Neuroinformatics Vol. 13, No. 2, 209-225, 2015.
[14]
Kamnitsas, K.; Ledig, C.; Newcombe, V. F. J.; Simpson, J. P.; Kane, A. D.; Menon, D. K.; Rueckert, D.; Glocker, B. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis Vol. 36, 61-78, 2017.
[15]
Kamnitsas, K.; Ferrante, E.; Parisot, S.; Ledig, C.; Nori, A. V.; Criminisi, A.; Rueckert, D.; Glocker, B.DeepMedic for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 10154. Crimi, A.; Menze, B.; Maier, O.; Reyes, M.; Winzeck, S.; Handels, H. Eds. Springer Cham, 138-149, 2016.
DOI
[16]
Chang, P. D.Fully convolutional deep residual neural networks for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 10154. Crimi, A.; Menze, B.; Maier, O.; Reyes, M.; Winzeck, S.; Handels, H. Eds. Springer Cham, 108-118, 2016.
[17]
Drozdzal, M.; Vorontsov, E.; Chartrand, G.; Kadoury, S.; Pal, C.The importance of skip connections in biomedical image segmentation. In: Deep Learning and Data Labeling for Medical Applications. Lecture Notes in Computer Science, Vol. 10008. Carneiro, G. et al. Eds. Springer Cham, 179-187, 2016.
DOI
[18]
Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S. S.; Brox, T.; Ronneberger, O.3D U-Net: Learning dense volumetric segmentation from sparse annotation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. Lecture Notes in Computer Science, Vol. 9901. Ourselin, S.; Joskowicz, L.; Sabuncu, M.; Unal, G.; Wells, W. Eds. Springer Cham, 424-432, 2016.
[19]
Lai, M. Deep learning for medical image segmentation. arXiv preprint arXiv:1505.02000, 2015.
[20]
Pereira, S.; Pinto, A.; Alves, V.; Silva, C. A. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging Vol. 35, No. 5, 1240-1251, 2016.
[21]
Tustison, N. J.; Avants, B. B.; Cook, P. A.; Zheng, Y. J.; Egan, A.; Yushkevich, P. A.; Gee, J. C. N4ITK: Improved N3 bias correction. IEEE Transactions on Medical Imaging Vol. 29, No. 6, 1310-1320, 2010.
[22]
Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 12, 2481-2495, 2017.
[23]
Noh, H.; Hong, S.; Han, B. Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, 1520-1528, 2015.
DOI
[24]
Bakas, S.; Akbari, H.; Sotiras, A.; Bilello, M.; Rozycki, M.; Kirby, J. S.; Freymann, J. B.; Farahani, K.; Davatzikos, C. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data Vol. 4, 170117, 2017.
[25]
Kamnitsas, K.; Bai, W.; Ferrante, E.; McDonagh, S.; Sinclair, M.; Pawlowski, N.; Rajchl, M.; Lee, M.; Kainz, B.; Rueckert, D.; Glocker, B.Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol 10670. Crimi, A.; Bakas, S.; Kuijf, H.; Menze, B.; Reyes, M. Eds. Springer Cham, 450-462, 2018.
DOI
[26]
Casamitjana, A.; Catà, M.; Sánchez, I.; Combalia, M.; Vilaplana, V.Cascaded V-Net using ROI masks for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science, Vol. 10670. Crimi, A.; Bakas, S.; Kuijf, H.; Menze, B.; Reyes, M. Eds. Springer Cham, 381-391, 2018.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 07 February 2019
Accepted: 24 March 2019
Published: 23 April 2019
Issue date: June 2019

Copyright

© The author(s) 2019

Acknowledgements

We would like to thank nVidia for their kind donation of a Titan XP GPU.

Rights and permissions

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from thecopyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return