多模态融合的深度学习脑肿瘤检测方法
|
摘要
针对目前传统方法脑肿瘤检测准确率低的问题,提出一种基于深度学习的三维脑肿瘤检测方法.首先将不同模态的脑肿瘤磁共振成像影像进行融合,获取不同模态下的脑肿瘤病灶三维空间特征;然后在卷积层和池化层之间增加实列归一化层,提高网络的收敛速度,缓解过拟合的问题;并对损失函数进行改进,采用加权损失函数加强对病灶区域的特征学习;最后结合后处理方法解决假阳脑肿瘤病灶多的问题.实验结果表明:提出的脑肿瘤检测方法可有效进行肿瘤病灶定位;相关性系数、敏感性和特异性三种评价指标分别达到了0.926 7、0.928 1和0.997 7,与二维检测网络相比,提高了4.6%、3.96%和0.04%,较初始的单模态脑肿瘤检测方法提升了13.2%、10.42%和0.12%.
Aiming at the low accuracy of traditional brain tumor detection,a three-dimensional brain tumor detection method based on deep learning was proposed.Firstly,the magnetic resonance images of different modal brain tumors were fused to obtain the three-dimensional features of brain tumor focus under different modalities.Then,an instance normalization layer was added between the convolution layer and the pooling layer to improve the convergence speed of the network and relieve the problem of overfitting.And the loss function was improved,the weighted loss function was used to enhance the feature learning of the focus area.Finally,the problem of more focuses in the false positive brain tumor was solved combining with the post-processing method.The experimental results show that the proposed brain tumor detection method can effectively detect the tumor focuses.The Dice coefficient,sensitivity and specificity of the three evaluation indexes reach 0.926 7,0.928 1 and 0.997 7 respectively.The three indicators improve 4.6%,3.96% and 0.04% compared with the 2 Ddetection network,and improve13.2%,10.42% and 0.12% compared with the initial single modal brain.
Aiming at the low accuracy of traditional brain tumor detection,a three-dimensional brain tumor detection method based on deep learning was proposed.Firstly,the magnetic resonance images of different modal brain tumors were fused to obtain the three-dimensional features of brain tumor focus under different modalities.Then,an instance normalization layer was added between the convolution layer and the pooling layer to improve the convergence speed of the network and relieve the problem of overfitting.And the loss function was improved,the weighted loss function was used to enhance the feature learning of the focus area.Finally,the problem of more focuses in the false positive brain tumor was solved combining with the post-processing method.The experimental results show that the proposed brain tumor detection method can effectively detect the tumor focuses.The Dice coefficient,sensitivity and specificity of the three evaluation indexes reach 0.926 7,0.928 1 and 0.997 7 respectively.The three indicators improve 4.6%,3.96% and 0.04% compared with the 2 Ddetection network,and improve13.2%,10.42% and 0.12% compared with the initial single modal brain.
引文
[1]WU Guo-qing,LI Ze-ju,WANG Yuan-yuan,et al.Primary central nervous system lymphoma and glioblastoma image differentiation based on sparse representation system[J].Journal of Biomedical Engineering,2018,35(5):754-760.吴国庆,李泽榉,汪源源,等.基于稀疏表示体系的原发性脑部淋巴瘤和胶质母细胞瘤图像鉴别[J].生物医学工程学杂志,2018,35(5):754-760.
[2]XIONG Jiao-jiao,LU Hong-yang,ZHANG Ming-hui,et al.Convolutional sparse coding in gradient domain for MRIreconstruction[J].Acta Automatica Sinica,2017,43(10):1841-1849.熊娇娇,卢红阳,张明辉,等.基于梯度域的卷积稀疏编码磁共振成像重建[J].自动化学报,2017,43(10):1841-1849.
[3]ZHANG Qiong-min,ZHANG Jing,WANG Min-tang,et al.Head and neck tumor segmentation based on augmented gradient level set method[J].Journal of Biomedical Engineering,2015,32(4):887-891+904.张琼敏,张劲,王敏堂,等.基于增强梯度水平集的头颈部肿瘤分割[J].生物医学工程学杂志,2015,32(4):887-891+904.
[4]ZHANG Teng-da,LV Xiao-qi,REN Xiao-yin,et al.Brain tumor MR image segmentation method based on fuzzy level set[J].Modern Electronics Technique,2016,39(18):91-95.张腾达,吕晓琪,任晓颖,等.基于模糊水平集的脑肿瘤MR图像分割方法[J].现代电子技术,2016,39(18):91-95.
[5]LI Na,XIONG Zhi-yong,XIE Jin,et al.Brain tumor segmentation on multi-modality magnetic resonance images based on tamura texture feature and SVM model[J].Journal of South-Central University for Nationalities(Natural Science Edition),2018,37(3):144-149.李娜,熊志勇,谢瑾,等.基于Tamura纹理特征提取和SVM的多模态脑肿瘤MR图像分割[J].中南民族大学学报(自然科学版),2018,37(3):144-149.
[6]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,39(6):1137-1149.
[7]YUAN Wen-hao,SUN Wen-zhu,XIA Bin,et al.Improving speech enhancement in unseen noise using deep convolutional neural network[J].Acta Automatica Sinica,2018,44(4):751-759.袁文浩,孙文珠,夏斌,等.利用深度卷积神经网络提高未知噪声下的语音增强性能[J].自动化学报,2018,44(4):751-759.
[8]XIAO Wen,YANG Lu,PAN Feng,et al.Automatic phase aberration compensation for digital holographic microscopy combined with phase fitting and deep learning[J].Acta Photonica Sinica,2018,47(12):1210001.肖文,杨璐,潘锋,等.结合划线拟合和深度学习的数字全息显微相位像差自动补偿方法[J].光子学报,2018,47(12):1210001.
[9]MA Hao-yu,XU Zhi-Hai,FENG Hua-jun,et al.Image super-resolution based on tiny recurrent convolutional neural network[J].Acta Photonica Sinica,2018,47(4):0410004.马昊宇,徐之海,冯华君,等.基于小递归卷积神经网络的图像超分辨算法[J].光子学报,2018,47(4):0410004.
[10]董荣凤.基于堆叠自动编码器的多模态脑肿瘤图像分割方法研究[D].成都:电子科技大学,2018.
[11]CHEN K,DING G,HAN J.Attribute-based supervised deep learning model for action recognition[J].Frontiers of Computer Science,2017,11(2):219-229.
[12]SHAN H,ZHANG Y,YANG Q,et al.3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network[J].IEEE Transactions on Medical Imaging,2018,37(6):1522.
[13]PEREIRA M B,WALLROTH M,JONSSON V,et al.Comparison of normalization methods for the analysis of metagenomic gene abundance data[J].Bmc Genomics,2018,19(1):274.
[14]HAN X,DAI Q.Batch-normalized Mlpconv-wise supervised pre-training network in network[J].Applied Intelligence,2017,48(7):142-155.
[15]BORGHAMMER P,JONSDOTTIR K Y,CUMMING P,et al.Normalization in PET group comparison studies-the importance of a valid reference region[J].Neuroimage,2008,40(2):529-540.
[16]LAGER C J,ESFANDIARI N H,SUBAUSTE A R,et al.Milestone weight loss goals(weight normalization and remission of obesity)after gastric bypass surgery:long-term results from the university of Michigan[J].Obesity Surgery,2017,27(7):1659-1666.
[17]ELAD M,MILANFAR P.Style transfer via texture synthesis[J].IEEE Transactions on Image Processing,2017,26(5):2338-2351.
[18]ZHAO X,WU Y,SONG G,et al.A deep learning model integrating FCNNs and CRFs for brain tumor segmentation[J].Medical Image Analysis,2017,43:98-111.
[19]JIANG N,WANG L.Quantum image scaling using nearest neighbor interpolation[J].Quantum Information Processing,2015,14(5):1559-1571.
[2]XIONG Jiao-jiao,LU Hong-yang,ZHANG Ming-hui,et al.Convolutional sparse coding in gradient domain for MRIreconstruction[J].Acta Automatica Sinica,2017,43(10):1841-1849.熊娇娇,卢红阳,张明辉,等.基于梯度域的卷积稀疏编码磁共振成像重建[J].自动化学报,2017,43(10):1841-1849.
[3]ZHANG Qiong-min,ZHANG Jing,WANG Min-tang,et al.Head and neck tumor segmentation based on augmented gradient level set method[J].Journal of Biomedical Engineering,2015,32(4):887-891+904.张琼敏,张劲,王敏堂,等.基于增强梯度水平集的头颈部肿瘤分割[J].生物医学工程学杂志,2015,32(4):887-891+904.
[4]ZHANG Teng-da,LV Xiao-qi,REN Xiao-yin,et al.Brain tumor MR image segmentation method based on fuzzy level set[J].Modern Electronics Technique,2016,39(18):91-95.张腾达,吕晓琪,任晓颖,等.基于模糊水平集的脑肿瘤MR图像分割方法[J].现代电子技术,2016,39(18):91-95.
[5]LI Na,XIONG Zhi-yong,XIE Jin,et al.Brain tumor segmentation on multi-modality magnetic resonance images based on tamura texture feature and SVM model[J].Journal of South-Central University for Nationalities(Natural Science Edition),2018,37(3):144-149.李娜,熊志勇,谢瑾,等.基于Tamura纹理特征提取和SVM的多模态脑肿瘤MR图像分割[J].中南民族大学学报(自然科学版),2018,37(3):144-149.
[6]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,39(6):1137-1149.
[7]YUAN Wen-hao,SUN Wen-zhu,XIA Bin,et al.Improving speech enhancement in unseen noise using deep convolutional neural network[J].Acta Automatica Sinica,2018,44(4):751-759.袁文浩,孙文珠,夏斌,等.利用深度卷积神经网络提高未知噪声下的语音增强性能[J].自动化学报,2018,44(4):751-759.
[8]XIAO Wen,YANG Lu,PAN Feng,et al.Automatic phase aberration compensation for digital holographic microscopy combined with phase fitting and deep learning[J].Acta Photonica Sinica,2018,47(12):1210001.肖文,杨璐,潘锋,等.结合划线拟合和深度学习的数字全息显微相位像差自动补偿方法[J].光子学报,2018,47(12):1210001.
[9]MA Hao-yu,XU Zhi-Hai,FENG Hua-jun,et al.Image super-resolution based on tiny recurrent convolutional neural network[J].Acta Photonica Sinica,2018,47(4):0410004.马昊宇,徐之海,冯华君,等.基于小递归卷积神经网络的图像超分辨算法[J].光子学报,2018,47(4):0410004.
[10]董荣凤.基于堆叠自动编码器的多模态脑肿瘤图像分割方法研究[D].成都:电子科技大学,2018.
[11]CHEN K,DING G,HAN J.Attribute-based supervised deep learning model for action recognition[J].Frontiers of Computer Science,2017,11(2):219-229.
[12]SHAN H,ZHANG Y,YANG Q,et al.3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network[J].IEEE Transactions on Medical Imaging,2018,37(6):1522.
[13]PEREIRA M B,WALLROTH M,JONSSON V,et al.Comparison of normalization methods for the analysis of metagenomic gene abundance data[J].Bmc Genomics,2018,19(1):274.
[14]HAN X,DAI Q.Batch-normalized Mlpconv-wise supervised pre-training network in network[J].Applied Intelligence,2017,48(7):142-155.
[15]BORGHAMMER P,JONSDOTTIR K Y,CUMMING P,et al.Normalization in PET group comparison studies-the importance of a valid reference region[J].Neuroimage,2008,40(2):529-540.
[16]LAGER C J,ESFANDIARI N H,SUBAUSTE A R,et al.Milestone weight loss goals(weight normalization and remission of obesity)after gastric bypass surgery:long-term results from the university of Michigan[J].Obesity Surgery,2017,27(7):1659-1666.
[17]ELAD M,MILANFAR P.Style transfer via texture synthesis[J].IEEE Transactions on Image Processing,2017,26(5):2338-2351.
[18]ZHAO X,WU Y,SONG G,et al.A deep learning model integrating FCNNs and CRFs for brain tumor segmentation[J].Medical Image Analysis,2017,43:98-111.
[19]JIANG N,WANG L.Quantum image scaling using nearest neighbor interpolation[J].Quantum Information Processing,2015,14(5):1559-1571.