Open Access
Issue
Published:
05 August 2019
Clinical Big Data and Deep Learning: Applications, Challenges, and Future Outlooks
Jianxin Wang(
)
)
Keywords:
deep learning, clinical data, Electronic Health Record (EHR), medical image, clinical note
Cite this article:
Yu Y, Li M, Liu L, et al.
Clinical Big Data and Deep Learning: Applications, Challenges, and Future Outlooks.
Big Data Mining and Analytics,
2019, 2(4): 288-305.
https://doi.org/10.26599/BDMA.2019.9020007
Download citation
1123
Views
65
Downloads
Citations
79
Crossref
42
WoS
99
Scopus
0
CSCD
The explosion of digital healthcare data has led to a surge of data-driven medical research based on machine learning. In recent years, as a powerful technique for big data, deep learning has gained a central position in machine learning circles for its great advantages in feature representation and pattern recognition. This article presents a comprehensive overview of studies that employ deep learning methods to deal with clinical data. Firstly, based on the analysis of the characteristics of clinical data, various types of clinical data (e.g., medical images, clinical notes, lab results, vital signs, and demographic informatics) are discussed and details provided of some public clinical datasets. Secondly, a brief review of common deep learning models and their characteristics is conducted. Then, considering the wide range of clinical research and the diversity of data types, several deep learning applications for clinical data are illustrated: auxiliary diagnosis, prognosis, early warning, and other tasks. Although there are challenges involved in applying deep learning techniques to clinical data, it is still worthwhile to look forward to a promising future for deep learning applications in clinical big data in the direction of precision medicine.
menu
Abstract
Full text
Outline
About this article
Clinical Big Data and Deep Learning: Applications, Challenges, and Future Outlooks
Jianxin Wang(
)
)
Abstract
The explosion of digital healthcare data has led to a surge of data-driven medical research based on machine learning. In recent years, as a powerful technique for big data, deep learning has gained a central position in machine learning circles for its great advantages in feature representation and pattern recognition. This article presents a comprehensive overview of studies that employ deep learning methods to deal with clinical data. Firstly, based on the analysis of the characteristics of clinical data, various types of clinical data (e.g., medical images, clinical notes, lab results, vital signs, and demographic informatics) are discussed and details provided of some public clinical datasets. Secondly, a brief review of common deep learning models and their characteristics is conducted. Then, considering the wide range of clinical research and the diversity of data types, several deep learning applications for clinical data are illustrated: auxiliary diagnosis, prognosis, early warning, and other tasks. Although there are challenges involved in applying deep learning techniques to clinical data, it is still worthwhile to look forward to a promising future for deep learning applications in clinical big data in the direction of precision medicine.
Keywords:
deep learning, clinical data, Electronic Health Record (EHR), medical image, clinical note
References(147)
[1]
I. Kurt, M. Ture, and A. T. Kurum, Comparing performances of logistic regression, classification and regression tree, and neural networks for predicting coronary artery disease, Expert Syst. Appl., vol. 34, no. 1, pp. 366-374, 2008.
[2]
L. V. Lita, S. P. Yu, S. Niculescu, and J. B. Bi, Large scale diagnostic code classification for medical patient records, in Proc. 3rd Int. Joint Conf. Natural Language Processing, Hyderabad, India, 2008, pp. 877-882.
[3]
B. Koopman, G. Zuccon, A. Nguyen, A. Bergheim, and N. Grayson, Automatic ICD-10 classification of cancers from free-text death certificates, Int. J. Med. Inform., vol. 84, no. 11, pp. 956-965, 2015.
[4]
D. X. Wang, X. Liu, and M. D. Wang, A DT-SVM strategy for stock futures prediction with big data, in Proc. 16th Int. Conf. Computational Science and Engineering, Sydney, Australia, 2013, pp. 1005-1012.
[5]
A. Casillas, K. Gojenola, A. Pérez, and M. Oronoz, Clinical text mining for efficient extraction of drug-allergy reactions, in Proc. 2016 IEEE Int. Conf. Bioinformatics and Biomedicine (BIBM), Shenzhen, China, 2016, pp. 946-952.
[6]
G. Moharasar and T. B. Ho, A semi-supervised approach for temporal information extraction from clinical text, in Proc. 2016 IEEE RIVF Int. Conf. Computing & Communication Technologies, Research, Innovation, and Vision for the Future (RIVF), Hanoi, Vietnam, 2016, pp. 7-12.
[7]
K. Orphanou, A. Stassopoulou, and E. Keravnou, Temporal abstraction and temporal Bayesian networks in clinical domains: A survey, Artif. Intell. Med., vol. 60, no. 3, pp. 133-149, 2014.
[8]
A. Subasi and M. I. Gursoy, EEG signal classification using PCA, ICA, LDA and support vector machines, Expert Syst. Appl., vol. 37, no. 12, pp. 8659-8666, 2010.
[9]
M. Pechenizkiy, A. Tsymbal, and S. Puuronen, PCA-based feature transformation for classification: Issues in medical diagnostics, in Proc. 17th IEEE Symp. Computer-Based Medical Systems, Bethesda, MD, USA, 2004, pp. 535-540.
[10]
A. Rumshisky, M. Ghassemi, T. Naumann, P. Szolovits, V. M. Castro, T. H. McCoy, and R. H. Perlis, Predicting early psychiatric readmission with natural language processing of narrative discharge summaries, Transl. Psychiatry, vol. 6, no. 10, p. e921, 2016.
[11]
Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, vol. 521, no. 7553, pp. 436-444, 2015.
[12]
A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, Corrigendum: Dermatologist-level classification of skin cancer with deep neural networks, Nature, vol. 546, no. 7660, p. 686, 2017.
[13]
A. Rajkomar, E. Oren, K. Chen, A. M. Dai, N. Hajaj, M. Hardt, P. J. Liu, X. B. Liu, J. Marcus, M. M. Sun, et al., Scalable and accurate deep learning for electronic health records, npj Digit. Med., vol. 1, no. 1, p. 18, 2018.
[14]
M. Zeng, M. Li, Z. H. Fei, F. X. Wu, Y. H. Li, Y. Pan, and J. X. Wang, A deep learning framework for identifying essential proteins by integrating multiple types of biological information, IEEE/ACM Transactions on Computational Biology and Bioinformatics, .
[15]
L. L. Liu, S. W. Chen, F. H. Zhang, F. X. Wu, Y. Pan, and J. X. Wang, Deep convolutional neural network for automatically segmenting acute ischemic stroke lesion in multi-modality MRI, Neural Comput. Appl., .
[16]
D. Ravì, C. Wong, F. Deligianni, M. Berthelot, J. Andreu-Perez, B. Lo, and G. Z. Yang, Deep learning for health informatics, IEEE J. Biomed. Health Inform., vol. 21, no. 1, pp. 4-21, 2017.
[17]
R. Fang, S. Pouyanfar, Y. M. Yang, S. C. Chen, and S. S. Iyengar, Computational health informatics in the big data age: A survey, ACM Comput. Surv., vol. 49, no. 1, p. 12, 2016.
[18]
D. G. Shen, G. R. Wu, and H. I. Suk, Deep learning in medical image analysis, Annu. Rev. Biomed. Eng., vol. 19, pp. 221-248, 2017.
[19]
J. G. Lee, S. Jun, Y. W. Cho, H. Lee, G. B. Kim, J. B. Seo, and N. Kim, Deep learning in medical imaging: General overview, Korean J. Radiol., vol. 18, no. 4, pp. 570-584, 2017.
[20]
J. M. Sun and C. K. Reddy, Big data analytics for healthcare, in Proc. 19th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Chicago, IL, USA, 2013, p. 1525.
[21]
A. Belle, R. Thiagarajan, S. M. R. Soroushmehr, F. Navidi, D. A. Beard, and K. Najarian, Big data analytics in healthcare, BioMed Res. Int., vol. 2015, p. 370194, 2015.
[22]
G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez, A survey on deep learning in medical image analysis, Med. Image Anal., vol. 42, pp. 60-88, 2017.
[23]
P. Burlina, D. E. Freund, N. Joshi, Y. Wolfson, and N. M. Bressler, Detection of age-related macular degeneration via deep learning, in Proc. 13th Int. Symp. Biomedical Imaging (ISBI), Prague, Czech Republic, 2016, pp. 184-188.
[24]
V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T. Madams, J. Cuadros, et al., Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs, JAMA, vol. 316, no. 22, pp. 2402-2410, 2016.
[25]
D. S. Kermany, M. Goldbaum, W. J. Cai, C. C. S. Valentim, H. Y. Liang, S. L. Baxter, A. McKeown, G. Yang, X. K. Wu, F. B. Tan, et al., Identifying medical diagnoses and treatable diseases by image-based deep learning, Cell, vol. 172, no. 5, pp. 1122-1131, 2018.
[26]
J. Liu, Y. Pan, M. Li, Z. Y. Chen, L. Tang, C. Q. Lu, and J. X. Wang, Applications of deep learning to MRI images: A survey, Big Data Mining and Analytics, vol. 1, no. 1, pp. 1-18, 2018.
[27]
T. Brosch, R. Tam, and Alzheimer’s Disease Neuroimaging Initiative, Manifold learning of brain MRIs by deep learning, in Proc. 16th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Nagoya, Japan, 2013, pp. 633-640.
[28]
S. M. Plis, D. R. Hjelm, R. Salakhutdinov, E. A. Allen, H. J. Bockholt, J. D. Long, H. J. Johnson, J. S. Paulsen, J. A. Turner, and V. D. Calhoun, Deep learning for neuroimaging: A validation study, Front. Neurosci., vol. 8, p. 229, 2014.
[29]
O. Emad, I. A. Yassine, and A. S. Fahmy, Automatic localization of the left ventricle in cardiac MRI images using deep learning, in Proc. 37th Annu. Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 2015, pp. 683-686.
[30]
B. Kong, Y. Q. Zhan, M. Shin, T. Denny, and S. T. Zhang, Recognizing end-diastole and end-systole frames via deep temporal regression network, in Proc. 19th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Athens, Greece, 2016, pp. 264-272.
[31]
T. A. Ngo, Z. Lu, and G. Carneiro, Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance, Med. Image Anal., vol. 35, pp. 159-171, 2017.
[32]
A. Jamaludin, T. Kadir, and A. Zisserman, SpineNet: Automatically pinpointing classification evidence in spinal MRIs, in Proc. 19th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Athens, Greece, 2016, pp. 949-954.
[33]
D. Forsberg, E. Sjöblom, and J. L. Sunshine, Detection and labeling of vertebrae in MR images using deep learning with clinical annotations as training data, J. Digit. Imaging, vol. 30, no. 4, pp. 406-412, 2017.
[34]
A. Prasoon, K. Petersen, C. Igel, F. Lauze, E. Dam, and M. Nielsen, Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network, in Proc. 16th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Nagoya, Japan, 2013, pp. 246-253.
[35]
P. J. Hu, F. Wu, J. L. Peng, Y. Y. Bao, F. Chen, and D. X. Kong, Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets, Int. J. Comput. Assist. Radiol. Surg., vol. 12, no. 3, pp. 399-411, 2017.
[36]
W. Thong, S. Kadoury, N. Piché, and C. J. Pal, Convolutional networks for kidney segmentation in contrast-enhanced CT scans, Comput. Methods Biomech. Biomed. Eng.: Imaging Vis., vol. 6, no. 3, pp. 277-282, 2018.
[37]
F. Lu, F. Wu, P. J. Hu, Z. Y. Peng, and D. X. Kong, Automatic 3D liver location and segmentation via convolutional neural network and graph cut, Int. J. Comput. Assist. Radiol. Surg., vol. 12, no. 2, pp. 171-182, 2017.
[38]
M. Anthimopoulos, S. Christodoulidis, L. Ebner, A. Christe, and S. Mougiakakou, Lung pattern classification for interstitial lung diseases using a deep convolutional neural network, IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1207-1216, 2016.
[39]
S. Christodoulidis, M. Anthimopoulos, L. Ebner, A. Christe, and S. Mougiakakou, Multisource transfer learning with convolutional neural networks for lung pattern analysis, IEEE J. Biomed. Health Inform., vol. 21, no. 1, pp. 76-84, 2017.
[40]
S. R. Tarando, C. Fetita, A. Faccinetto, and P. Y. Brillet, Increasing CAD system efficacy for lung texture analysis using a convolutional network, in Proc. Medical Imaging 2016: Computer-Aided Diagnosis, San Diego, CA, USA, 2016, p. 97850Q.
[41]
Y. Bar, I. Diamant, L. Wolf, and H. Greenspan, Deep learning with non-medical training used for chest pathology identification, in Proc. Medical Imaging 2015: Computer-Aided Diagnosis, Orlando, FL, USA, 2015, p. 94140V.
[42]
M. Cicero, A. Bilbily, E. Colak, T. Dowdell, B. Gray, K. Perampaladas, and J. Barfett, Training and validating a deep convolutional neural network for computer-aided detection and classification of abnormalities on frontal chest radiographs, Invest. Radiol., vol. 52, no. 5, pp. 281-287, 2017.
[43]
S. Hwang, H. E. Kim, J. Jeong, and H. J. Kim, A novel approach for tuberculosis screening based on deep convolutional neural networks, in Proc. Medical Imaging 2016: Computer-Aided Diagnosis, San Diego, CA, USA, 2016, p. 97852W.
[44]
A. Dubrovina, P. Kisilev, B. Ginsburg, S. Hashoul, and R. Kimmel, Computational mammography using deep neural networks, Comput. Methods Biomech. Biomed. Eng.: Imaging Vis., vol. 6, no. 3, pp. 243-247, 2018.
[45]
T. Kooi, B. van Ginneken, N. Karssemeijer, and A. den Heeten, Discriminating solitary cysts from soft tissue lesions in mammography using a pretrained deep convolutional neural network, Med. Phys., vol. 44, no. 3, pp. 1017-1027, 2017.
[46]
T. Kooi, G. Litjens, B. van Ginneken, A. Gubern-Mérida, C. I. Sánchez, R. Mann, A. den Heeten, and N. Karssemeijer, Large scale deep learning for computer aided detection of mammographic lesions, Med. Image Anal., vol. 35, pp. 303-312, 2017.
[47]
A. Perotte, R. Pivovarov, K. Natarajan, N. Weiskopf, F. Wood, and N. Elhadad, Diagnosis code assignment: Models and evaluation metrics, J. Am. Med. Inform. Assoc., vol. 21, no. 2, pp. 231-237, 2014.
[48]
M. Li, Z. H. Fei, M. Zeng, F. X. Wu, Y. H. Li, Y. Pan, and J. X. Wang, Automated ICD-9 coding via a deep learning approach, IEEE/ACM Transactions on Computational Biology and Bioinformatics, .
[49]
H. R. Shi, P. T. Xie, Z. T. Hu, M. Zhang, and E. P. Xing, Towards automated ICD coding using deep learning, arXiv preprint arXiv: 1711.04075, 2017.
[50]
T. Baumel, J. Nassour-Kassis, R. Cohen, M. Elhadad, and N. Elhadad, Multi-label classification of patient notes a case study on ICD code assignment, arXiv preprint arXiv: 1709.09587, 2017.
[51]
M. Zeng, M. Li, Z. H. Fei, Y. Yu, Y. Pan, and J. X. Wang, Automatic ICD-9 coding via deep transfer learning, Neurocomputing, vol. 324, pp. 43-50, 2019.
[52]
S. Gehrmann, F. Dernoncourt, Y. R. Li, E. T. Carlson, J. T. Wu, J. Welt, J. Jr. Foote, E. T. Moseley, D. W. Grant, P. D. Tyler, et al. , Comparing rule-based and deep learning models for patient phenotyping, arXiv preprint arXiv: 1703.08705, 2017.
[53]
S. Gao, M. T. Young, J. X. Qiu, H. J. Yoon, J. B. Christian, P. A. Fearn, G. D. Tourassi, and A. Ramanthan, Hierarchical attention networks for information extraction from cancer pathology reports, J. Am. Med. Inform. Assoc., vol. 25, no. 3, pp. 321-330, 2018.
[54]
H. C. Shin, L. Lu, L. Kim, A. Seff, J. H. Yao, and R. M. Summers, Interleaved text/image Deep Mining on a large-scale radiology database for automated image interpretation, J. Mach. Learn. Res., vol. 17, no. 1, pp. 3729-3759, 2016.
[55]
L. Dai, R. G. Fang, H. T. Li, X. H. Hou, B. Sheng, Q. Wu, and W. P. Jia, Clinical report guided retinal microaneurysm detection with multi-sieving deep learning, IEEE Trans. Med. Imaging, vol. 37, no. 5, pp. 1149-1161, 2018.
[56]
F. Duarte, B. Martins, C. S. Pinto, and M. J. Silva, A deep learning method for ICD-10 coding of free-text death certificates, in Proc. 18th EPIA Conf. Artificial Intelligence, Porto, Portugal, 2017, pp. 137-149.
[57]
A. Perotte, R. Ranganath, J. S. Hirsch, D. Blei, and N. Elhadad, Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis, J. Am. Med. Inform. Assoc., vol. 22, no. 4, pp. 872-880, 2015.
[58]
M. Carrara, G. Baselli, and M. Ferrario, Mortality prediction in septic shock patients: Towards new personalized models in critical care, in Proc. 37th Annu. Int. Conf. of the Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 2015, pp. 2792-2795.
[59]
M. M. Ghassemi, S. E. Richter, I. M. Eche, T. W. Chen, J. Danziger, and L. A. Celi, A data-driven approach to optimized medication dosing: A focus on heparin, Intensive Care Med., vol. 40, no. 9, pp. 1332-1339, 2014.
[60]
S. Nemati, M. M. Ghassemi, and G. D. Clifford, Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach, in Proc. 38th Annu. Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2016, pp. 2978-2981.
[61]
M. Dunitz, G. Verghese, and T. Heldt, Predicting hyperlactatemia in the MIMIC II database, in Proc. 37th Annu. Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 2015, pp. 985-988
[62]
K. Gunnarsdottir, V. Sadashivaiah, M. Kerr, S. Santaniello, and S. V. Sarma, Using demographic and time series physiological features to classify sepsis in the intensive care unit, in Proc. 38th Annu. Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2016, pp. 778-782.
[63]
A. Lanata, G. Valenza, M. Nardelli, C. Gentili, and E. P. Scilingo, Complexity index from a personalized wearable monitoring system for assessing remission in mental health, IEEE J. Biomed. Health Inform., vol. 19, no. 1, pp. 132-139, 2015.
[64]
L. W. Li, R. P. Adams, L. Mayaud, G. B. Moody, A. Malhotra, R. G. Mark, and S. Nemati, A physiological time series dynamics-based approach to patient monitoring and outcome prediction, IEEE J. Biomed. Health Inform., vol. 19, no. 3, pp. 1068-1076, 2015.
[65]
Z. Feng, R. R. Bhat, X. Y. Yuan, D. Freeman, T. Baslanti, A. Bihorac, and X. L. Li, Intelligent perioperative system: Towards real-time big data analytics in surgery risk assessment, in 15th Intl Conf. on Dependable, Autonomic and Secure Computing, 15th Intl Conf. on Pervasive Intelligence and Computing, 3rd Intl Conf. on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC / PiCom / DataCom / CyberSciTech), Orlando, FL, USA, 2017, pp. 1254-1259.
[66]
P. X. Cheng, P. Yin, P. S. Ning, L. J. Wang, X. J. Cheng, Y. N. Liu, D. C. Schwebel, J. M. Liu, J. L. Qi, G. Q. Hu, et al., Trends in traumatic brain injury mortality in China, 2006-2013: A population-based longitudinal study, PLoS Med., vol. 14, no. 7, p. e1002332, 2017.
[67]
Healthcare Cost and Utilization Project (HCUP), https://www.hcup-us.ahrq.gov, 2019.
[68]
F. O. Otite, P. Khandelwal, A. M. Malik, S. Chaturvedi, R. L. Sacco, and J. G. Romano, Ten-year temporal trends in medical complications after acute intracerebral hemorrhage in the United States, Stroke, vol. 48, no. 3, pp. 596-603, 2017.
[69]
J. Y. Zhou, J. Liu, V. A. Narayan, and J. P. Ye, Modeling disease progression via fused sparse group lasso, in Proc. 18th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Beijing, China, 2012, pp. 1095-1103.
[70]
J. Liu, J. X. Wang, Z. J. Tang, B. Hu, F. X. Wu, and Y. Pan, Improving Alzheimer’s disease classification by combining multiple measures, IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 15, no. 5, pp. 1649-1659, 2018.
[71]
L. L. Liu, J. X. Wang, M. Li, F. X. Wu, H. D. Li, Y. Yu, and Z. H. Fei, An interpretable model for predicting side effects of analgesics for osteoarthritis, in Proc. 2017 IEEE Int. Conf. Bioinformatics and Biomedicine (BIBM), Kansas City, MO, USA, 2017, pp. 861-864.
[72]
D. C. Mackey, L. Y. Lui, P. M. Cawthon, K. Ensrud, K. Yaffe, and S. R. Cummings, Life-space mobility and mortality in older women: Prospective results from the study of osteoporotic fractures, J. Am. Geriatr. Soc., vol. 64, no. 11, pp. 2226-2234, 2016.
[73]
Grand-Challenges, Grand challenges in biomedical image analysis, http:www.grand-challenge.org, 2019.
[74]
Ö. Uzuner, B. R. South, S. Y. Shen, and S. L. DuVall, 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text, J. Am. Med. Inform. Assoc., vol. 18, no. 5, pp. 552-556, 2011.
[75]
Ö. Uzuner, A. Bodnari, S. Y. Shen, T. Forbush, J. Pestian, and B. R. South, Evaluating the state of the art in coreference resolution for electronic medical records, J. Am. Med. Inf. Assoc., vol. 19, no. 5, pp. 786-791, 2012.
[76]
W. Y. Sun, A. Rumshisky, and Ö. Uzuner, Evaluating temporal relations in clinical text: 2012 i2b2 challenge, J. Am. Med. Inform. Assoc., vol. 20, no. 5, pp. 806-813, 2013.
[77]
A. Stubbs and Ö. Uzuner, Annotating risk factors for heart disease in clinical narratives for diabetic patients, J. Biomed. Inform., vol. 58, no. Suppl, pp. S78-S91, 2015.
[78]
M. Filannino, A. Stubbs, and Ö. Uzuner, Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks track 2, J. Biomed. Inform., vol. 75, no. Suppl, pp. S62-S70, 2017.
[79]
G. B. Moody and R. G. Mark, A database to support development and evaluation of intelligent intensive care monitoring, in Proc. 1996 Computers in Cardiology, Indianapolis, IN, USA, 1996, pp. 657-660.
[80]
M. Saeed, M. Villarroel, A. T. Reisner, G. Clifford, L. W. Lehman, G. Moody, T. Heldt, T. H. Kyaw, B. Moody, and R. G. Mark, Multiparameter intelligent monitoring in intensive care II: A public-access intensive care unit database, Crit. Care Med., vol. 39, no. 5, pp. 952-960, 2011.
[81]
J. Lee, D. J. Scott, M. Villarroel, G. D. Clifford, M. Saeed, and R. G. Mark, Open-access MIMIC-II database for intensive care research, in Proc. 2011 Annu. Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Boston, MA, USA, 2011, pp. 8315-8318.
[82]
R. Miotto, L. Li, B. A. Kidd, and J. T. Dudley, Deep patient: An unsupervised representation to predict the future of patients from the electronic health records, Sci. Rep., vol. 6, p. 26094, 2016.
[83]
E. Choi, M. T. Bahadori, E. Searles, C. Coffey, M. Thompson, J. Bost, J. Tejedor-Sojo, and J. M. Sun, Multi-layer representation learning for medical concepts, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 1495-1504.
[84]
Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time series, in Handbook of Brain Theory and Neural Networks, M. A. Arbib, ed. Cambridge, MA, USA: MIT Press, 1995.
[85]
W. F. Yin, X. Z. Yang, L. Zhang, and E. Oki, ECG monitoring system integrated with IR-UWB radar based on CNN, IEEE Access, vol. 4, pp. 6344-6351, 2016.
[86]
H. Z. Fu, J. Cheng, Y. W. Xu, D. W. K. Wong, J. Liu, and X. C. Cao, Joint optic disc and cup segmentation based on multi-label deep network and polar transformation, IEEE Trans. Med. Imaging, vol. 37, no. 7, pp. 1597-1605, 2018.
[87]
F. Milletari, N. Navab, and S. A. Ahmadi, V-Net: Fully convolutional neural networks for volumetric medical image segmentation, in Proc. 4th Int. Conf. 3D Vision (3DV), Stanford, CA, USA, 2016, pp. 565-571.
[88]
A. Fakhry, T. Zeng, and S. W. Ji, Residual deconvolutional networks for brain electron microscopy image segmentation, IEEE Trans. Med. Imaging, vol. 36, no. 2, pp. 447-456, 2017.
[89]
R. Guerrero, C. Qin, O. Oktay, C. Bowles, L. Chen, R. Joules, R. Wolz, M. C. Valdés-Hernández, D. A. Dickie, J. Wardlaw, et al., White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks, NeuroImage Clin., vol. 17, pp. 918-934, 2018.
[90]
D. M. Pelt and J. A. Sethian, A mixed-scale dense convolutional neural network for image analysis, Proc. Natl. Acad. Sci. USA, vol. 115, no. 2, pp. 254-259, 2018.
[91]
G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, Densely connected convolutional networks, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2261-2269.
[92]
Q. L. Zhu, X. L. Li, A. Conesa, and C. Pereira, GRAM-CNN: A deep learning approach with local context for named entity recognition in biomedical text, Bioinformatics, vol. 34, no. 9, pp. 1547-1554, 2018.
[93]
K. Liu, S. W. Peng, J. Q. Wu, C. X. Zhai, H. Mamitsuka, and S. F. Zhu, MeSHLabeler: Improving the accuracy of large-scale MeSH indexing by integrating diverse evidence, Bioinformatics, vol. 31, no. 12, pp. i339-i347, 2015.
[94]
S. W. Peng, R. H. You, H. N. Wang, C. X. Zhai, H. Mamitsuka, and S. F. Zhu, DeepMeSH: Deep semantic representation for improving large-scale MeSH indexing, Bioinformatics, vol. 32, no. 12, pp. i70-i79, 2016.
[95]
S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997.
[96]
E. Choi, M. T. Bahadori, A. Schuetz, W. F. Stewart, and J. M. Sun, Doctor AI: Predicting clinical events via recurrent neural networks, in Proc. 2016 Machine Learning for Healthcare Conf., Los Angeles, CA, USA, 2016, pp. 301-318.
[97]
Y. H. Wu, M. Jiang, J. B. Lei, and H. Xu, Named entity recognition in Chinese clinical text using deep neural network, Stud. Health Technol. Inform., vol. 216, pp. 624-628, 2015.
[98]
A. N. Jagannatha and H. Yu, Bidirectional RNN for medical event detection in electronic health records, in Proc. Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA, 2016, pp. 473-482.
[99]
A. Khatami, A. Khosravi, T. Nguyen, C. P. Lim, and S. Nahavandi, Medical image analysis using wavelet transform and deep belief networks, Exp. Syst. Appl., vol. 86, pp. 190-198, 2017.
[100]
K. L. Hua, C. H. Hsu, S. C. Hidayati, W. H. Cheng, and Y. J. Chen, Computer-aided classification of lung nodules on computed tomography images via deep learning technique, Onco Targets Ther., vol. 8, pp. 2015-2022, 2015.
[101]
G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science, vol. 313, no. 5786, pp. 504-507, 2006.
[102]
H. I. Suk and D. G. Shen, Deep learning-based feature representation for AD/MCI classification, in Proc. 16th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Nagoya, Japan, 2013, pp. 583-590.
[103]
S. Sharma, I. Umar, L. Ospina, D. Wong, and H. R. Tizhoosh, Stacked autoencoders for medical image search, in Proc. 12th Int. Symp. Visual Computing, Las Vegas, NV, USA, 2016, pp. 45-54.
[104]
J. X. Chen, L. Yang, Y. Z. Zhang, M. Alber, and D. Z. Chen, Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation, in Proc. 30th Conf. Neural Information Processing Systems (NIPS), Barcelona, Spain, 2016, pp. 3036-3044.
[105]
G. S. Birkhead, M. Klompas, and N. R. Shah, Uses of electronic health records for public health surveillance to advance public health, Annu. Rev. Public Health, vol. 36, pp. 345-359, 2015.
[106]
F. L. Ma, R. Chitta, J. Zhou, Q. Z. You, T. Sun, and J. Gao, Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks, in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Halifax, Canada, 2017, pp. 1903-1911.
[107]
A. Prakash, S. Y. Zhao, S. A. Hasan, V. Datla, K. Lee, A. Qadir, J. Liu, and O. Farri, Condensed memory networks for clinical diagnostic inferencing, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3274-3280.
[108]
E. Choi, M. T. Bahadori, J. A. Kulas, A. Schuetz, W. F. Stewart, and J. M. Sun, RETAIN: An interpretable predictive model for healthcare using reverse time attention mechanism, in Proc. 30th Int. Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 3504-3512.
[109]
F. Duarte, B. Martins, C. S. Pinto, and M. J. Silva, Deep neural models for ICD-10 coding of death certificates and autopsy reports in free-text, J. Biomed. Inform., vol. 80, pp. 64-77, 2018.
[110]
D. L. Guo, M. Li, Y. Yu, Y. H. Li, G. H. Duan, F. X. Wu, and J. X. Wang, Disease inference with symptom extraction and bidirectional recurrent neural network, in Proc. 2018 Int. Conf. Bioinformatics and Biomedicine (BIBM), Madrid, Spain, 2018, pp. 864-868
[111]
Y. Yu, M. Li, L. L. Liu, Z. H. Fei, F. X. Wu, and J. X. Wang, Automatic ICD code assignment of Chinese clinical notes based on multilayer attention BiRNN, J. Biomed. Inform., vol. 91, p. 103114, 2019.
[112]
R. Poplin, A. V. Varadarajan, K. Blumer, Y. Liu, M. V. McConnell, G. S. Corrado, L. Peng, and D. R. Webster, Predicting cardiovascular risk factors from retinal fundus photographs using deep learning, arXiv preprint arXiv: 1708.09843, 2017.
[113]
H. Li, X. Y. Li, M. Ramanathan, and A. D. Zhang, Prediction and informative risk factor selection of bone diseases, IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 12, no. 1, pp. 79-91, 2015.
[114]
J. Donzé, D. Aujesky, D. Williams, and J. L. Schnipper, Potentially avoidable 30-day hospital readmissions in medical patients: Derivation and validation of a prediction model, JAMA Intern. Med., vol. 173, no. 8, pp. 632-638, 2013.
[115]
J. Futoma, J. Morris, and J. Lucas, A comparison of models for predicting early hospital readmissions, J. Biomed. Inform., vol. 56, pp. 229-238, 2015.
[116]
P. Nguyen, T. Tran, N. Wickramasinghe, and S. Venkatesh, Deepr: A convolutional net for medical records, IEEE J. Biomed. Health Inform., vol. 21, no. 1, pp. 22-30, 2017.
[117]
T. Pham, T. Tran, D. Phung, and S. Venkatesh, DeepCare: A deep dynamic memory model for predictive medicine, in Proc. 20th Pacific-Asia Conf. Knowledge Discovery and Data Mining, Auckland, New Zealand, 2016, pp. 30-41.
[118]
P. Grnarova, F. Schmidt, S. L. Hyland, and C. Eickhoff, Neural document embeddings for intensive care patient mortality prediction, arXiv preprint arXiv: 1612.00467, 2016.
[119]
G. Carneiro, L. Oakden-Rayner, A. P. Bradley, J. Nascimento, and L. Palmer, Automated 5-year mortality prediction using deep learning and radiomics features from chest computed tomography, in Proc. 14th Int. Symp. Biomedical Imaging (ISBI), Melbourne, Australia, 2017, pp. 130-134.
[120]
H. K. van der Burgh, R. Schmidt, H. J. Westeneng, M. A. de Reus, L. H. van den Berg, and M. P. van den Heuvel, Deep learning predictions of survival based on MRI in amyotrophic lateral sclerosis, NeuroImage Clin., vol. 13, pp. 361-369, 2017.
[121]
R. H. Ju, C. H. Hu, P. Zhou, and Q. Z. Li, Early diagnosis of Alzheimer’s disease based on resting-state brain networks and deep learning, IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 16, no. 1, pp. 244-257, 2019.
[122]
D. H. Lu, K. Popuri, G. W. Ding, R. Balachandar, M. F. Beg, and Alzheimer’s Disease Neuroimaging Initiative, Multimodal and multiscale deep neural networks for the early diagnosis of Alzheimer’s disease using structural MR and FDG-PET images, Sci. Rep., vol. 8, p. 5697, 2018.
[123]
E. Choi, A. Schuetz, W. F. Stewart, and J. M. Sun, Using recurrent neural network models for early detection of heart failure onset, J. Am. Med. Inform. Assoc., vol. 24, no. 2, pp. 361-370, 2017.
[124]
Y. Luo, Recurrent neural networks for classifying relations in clinical notes, J. Biomed. Inform., vol. 72, pp. 85-95, 2017.
[125]
Z. J. Liu, B. Z. Tang, X. L. Wang, and Q. C. Chen, De-identification of clinical notes via recurrent neural network and conditional random field, J. Biomed. Inform., vol. 75 Suppl, pp. S34-S42, 2017.
[126]
F. Dernoncourt, J. Y. Lee, Ö. Uzuner, and P. Szolovits, De- identification of patient notes with recurrent neural networks, J. Am. Med. Inform. Assoc., vol. 24, no. 3, pp. 596-606, 2017.
[127]
C. Payer, D. Štern, H. Bischof, and M. Urschler, Regressing heatmaps for multiple landmark localization using CNNs, in Proc. 19th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Athens, Greece, 2016, pp. 230-238.
[128]
B. D. de Vos, J. M. Wolterink, P. A. de Jong, M. A. Viergever, and I. Išgum, 2D image classification for 3D anatomy localization: Employing deep convolutional neural networks, in Proc. Medical Imaging 2016: Image Processing, San Diego, CA, USA, 2016, p. 97841Y.
[129]
Q. Dou, H. Chen, L. Q. Yu, L. Zhao, J. Qin, D. F. Wang, V. C. T. Mok, L. Shi, and P. A. Heng, Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks, IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1182-1195, 2016.
[130]
X. R. Liu, H. R. Tizhoosh, and J. Kofman, Generating binary tags for fast medical image retrieval based on convolutional nets and Radon Transform, in Proc. 2016 Int. Joint Conf. Neural Networks (IJCNN), Vancouver, Canada, 2016, pp. 2872-2878.
[131]
A. Shah, S. Conjeti, N. Navab, and A. Katouzian, Deeply learnt hashing forests for content based image retrieval in prostate MR images, in Proc. Medical Imaging 2016: Image Processing, San Diego, CA, USA, 2016, p. 978414.
[132]
M. J. J. P. van Grinsven, B. van Ginneken, C. B. Hoyng, T. Theelen, and C. I. Sánchez, Fast convolutional neural network training using selective data sampling: Application to hemorrhage detection in color fundus images, IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1273-1284, 2016.
[133]
T. D. Vu, H. J. Yang, V. Q. Nguyen, A. R. Oh, and M. S. Kim, Multimodal learning using convolution neural network and Sparse Autoencoder, in Proc. 2017 Int. Conf. Big Data and Smart Computing (BigComp), Jeju, South Korea, 2017, pp. 309-312.
[134]
D. H. Lu, K. Popuri, G. W. Ding, R. Balachandar, and M. F. Beg, Multiscale deep neural network based analysis of FDG-PET images for the early diagnosis of Alzheimer’s disease, Med. Image Anal., vol. 46, pp. 26-34, 2018.
[135]
D. N. Cheng and M. H. Liu, Combining convolutional and recurrent neural networks for Alzheimer’s disease diagnosis using PET images, in Proc. 2017 IEEE Int. Conf. Imaging Systems and Techniques (IST), Beijing, China, 2017, pp. 1-5.
[136]
A. Ortiz, J. Munilla, J. M. Górriz, and J. Ramírez, Ensembles of deep learning architectures for the early diagnosis of the Alzheimer’s disease, Int. J. Neural Syst., vol. 26, no. 7, p. 1650025, 2016.
[137]
A. Payan and G. Montana, Predicting Alzheimer’s disease: A neuroimaging study with 3D convolutional neural networks, arXiv preprint arXiv: 1502.02506, 2015.
[138]
J. Shi, X. Zheng, Y. Li, Q. Zhang, and S. H. Ying, Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of Alzheimer’s disease, IEEE J. Biomed. Health Inform., vol. 22, no. 1, pp. 173-183, 2018.
[139]
A. Suzani, A. Rasoulian, A. Seitel, S. Fels, R. N. Rohling, and P. Abolmaesumi, Deep learning for automatic localization, identification, and segmentation of vertebral bodies in volumetric MR images, in Proc. Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, Orlando, FL, USA, 2015, pp. 773-778.
[140]
H. R. Roth, L. Lu, A. Farag, A. Sohn, and R. M. Summers, Spatial aggregation of holistically-nested networks for automated pancreas segmentation, in Proc. 19th Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Athens, Greece, 2016, pp. 451-459.
[141]
J. J. Nappi, T. Hironaka, D. Regge, and H. Yoshida, Deep transfer learning of virtual endoluminal views for the detection of polyps in CT colonography, in Proc. Medical Imaging 2016: Computer-Aided Diagnosis, San Diego, CA, USA, 2016, p. 97852B.
[142]
G. van Tulder and M. de Bruijne, Combining generative and discriminative representation learning for lung CT analysis with convolutional restricted Boltzmann machines, IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1262-1272, 2016.
[143]
J. Wang, H. J. Ding, F. M. Azamian, B. Zhou, C. Iribarren, S. Molloi, and P. Baldi, Detecting cardiovascular disease from mammograms with deep learning, IEEE Trans. Med. Imaging, vol. 36, no. 5, pp. 1172-1181, 2017.
[144]
A. Rajkomar, S. Lingam, A. G. Taylor, M. Blum, and J. Mongan, High-throughput classification of radiographs using deep convolutional neural networks, J. Digit. Imaging, vol. 30, no. 1, pp. 95-101, 2017.
[145]
M. Kallenberg, K. Petersen, M. Nielsen, A. Y. Ng, P. F. Diao, C. Igel, C. M. Vachon, K. Holland, R. R. Winkel, N. Karssemeijer, et al., Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring, IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1322-1331, 2016.
[146]
L. L. Liu, Y. Yu, Z. H. Fei, M. Li, F. X. Wu, H. D. Li, Y. Pan, and J. X. Wang, An interpretable boosting model to predict side effects of analgesics for osteoarthritis, BMC Syst. Biol., vol. 12, no. Suppl 6, p. 105, 2018.
[147]
The White House, Precision medicine: Health care tailored to you, https://obamawhitehouse.archives.gov/blog/2016/02/25/precision-medicine-health-care-tailored-you, 2016.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 08 March 2019
Accepted: 15 March 2019
Published:
05 August 2019
Issue date: December 2019
Copyright
© The author(s) 2019
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (Nos. 61772552 and 61772557), the 111 Project (No. B18059), and the Hunan Provincial Science and Technology Program (No. 2018WK4001).
Rights and permissions
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
FEEDBACK 
TOP