Journal Home > Volume 24 , Issue 2
Tsinghua Science and Technology 2019, 24(2): 195-206 https://doi.org/10.26599/TST.2018.9010074
Open Access | Issue | Published: 31 December 2018

Parallel ADR Detection Based on Spark and BCPNN

Show Author's Information Li Sun,  Shan Sun,  Tianlei Wang,  Jiyun Li( ),  Jingsheng Lin
School of Computer Science and Technology, Donghua University, Shanghai 201620, China.
Ruijin Hospital Affiliated to Shanghai Jiao Tong University, Shanghai 200020, China.
Keywords:
Adverse Drug Reaction (ADR), Bayesian Confidence Propagation Neural Network (BCPNN), parallel, Spark
Cite this article:
Sun L, Sun S, Wang T, et al. Parallel ADR Detection Based on Spark and BCPNN. Tsinghua Science and Technology, 2019, 24(2): 195-206. https://doi.org/10.26599/TST.2018.9010074
Download citation
EndNote(RIS)
BibTeX

493

Views

38

Downloads

Citations

4

Crossref

N/A

WoS

6

Scopus

1

CSCD

Abstract Full text About this article

Adverse Drug Reaction (ADR) is one of the major challenges to the evaluation of drug safety in the medical field. The Bayesian Confidence Propagation Neural Network (BCPNN) algorithm is the main algorithm used by the World Health Organization to monitor ADRs. Currently, ADR reports are collected through the spontaneous reporting system. However, with the continuous increase in ADR reports and possible use scenarios, the efficiency of the stand-alone ADR detection algorithm will encounter considerable challenges. Meanwhile, the BCPNN algorithm requires a certain number of disk I/O, which leads to considerable time consumption. In this study, we propose a Spark-based parallel BCPNN algorithm, which speeds up data processing and reduces the number of disk I/O in BCPNN, and two optimization strategies. Then, the ADR data collected from the FDA Adverse Event Reporting System are used to verify the performance of the proposed algorithm and its optimization strategies. Experiments show that the parallel BCPNN can significantly accelerate data processing and the optimized algorithm has a high acceleration rate and can effectively prevent memory overflow. Finally, we apply the proposed algorithm to a dataset provided by a real medical consortium. Experiments further prove the performance and practical value of the proposed algorithm.


menu
Abstract
Full text
Outline
About this article

Parallel ADR Detection Based on Spark and BCPNN

Show Author's information Li Sun, Shan Sun, Tianlei Wang, Jiyun Li( ), Jingsheng Lin
School of Computer Science and Technology, Donghua University, Shanghai 201620, China.
Ruijin Hospital Affiliated to Shanghai Jiao Tong University, Shanghai 200020, China.

Abstract

Adverse Drug Reaction (ADR) is one of the major challenges to the evaluation of drug safety in the medical field. The Bayesian Confidence Propagation Neural Network (BCPNN) algorithm is the main algorithm used by the World Health Organization to monitor ADRs. Currently, ADR reports are collected through the spontaneous reporting system. However, with the continuous increase in ADR reports and possible use scenarios, the efficiency of the stand-alone ADR detection algorithm will encounter considerable challenges. Meanwhile, the BCPNN algorithm requires a certain number of disk I/O, which leads to considerable time consumption. In this study, we propose a Spark-based parallel BCPNN algorithm, which speeds up data processing and reduces the number of disk I/O in BCPNN, and two optimization strategies. Then, the ADR data collected from the FDA Adverse Event Reporting System are used to verify the performance of the proposed algorithm and its optimization strategies. Experiments show that the parallel BCPNN can significantly accelerate data processing and the optimized algorithm has a high acceleration rate and can effectively prevent memory overflow. Finally, we apply the proposed algorithm to a dataset provided by a real medical consortium. Experiments further prove the performance and practical value of the proposed algorithm.

Keywords: Adverse Drug Reaction (ADR), Bayesian Confidence Propagation Neural Network (BCPNN), parallel, Spark

References(24)

[1]
Bate A., Lindquist M., Edwards I. R., and Orre R., A data mining approach for signal detection and analysis, Drug Saf., vol. 25, no. 6, pp. 393-397, 2002.
DOI Google Scholar
[2]
Bate A., The use of a Bayesian confidence propagation neural network in pharmacovigilance, PhD dissertation, Umeå University, Sweden, 2003.
[3]
Karimi S., Wang C., Metke-Jimenez A., Gaire R., and Paris C., Text and data mining techniques in adverse drug reaction detection, ACM Computing Surveys, vol. 47, no. 4, p. 56, 2015.
DOI Google Scholar
[4]
Chen W. G. and Deng J. X., A study on signal detection and automatic warning algorithm for adverse drug reaction, in Proc. 2008 International Conference on Computer Science and Software Engineering, 2008.
[5]
Swetha K. V., Sathyadevan S., and Bilna P., Network data analysis using spark, in Software Engineering in Intelligent Systems, Silhavy R., Senkerik R., Oplatkova Z., Prokopova Z., and Silhavy P., eds. Springer, 2015, pp. 253-259.
DOI
[6]
Bate A., Lindquist M., Edwards I. R., Olsson S., Orre R., Lansner A., and De Freitas R. M., A Bayesian neural network method for adverse drug reaction signal generation, Eur. J. Clin. Pharmacol., vol. 54, no. 4, pp. 315-321, 1998.
DOI Google Scholar
[7]
Farahini N., Hemani A., Lansner A., Clermidy F., and Svensson C., A scalable custom simulation machine for the Bayesian Confidence Propagation Neural Network model of the brain. in Proc. 2014 19th Asia and South Pacific Design Automation Conf., Singapore, 2014, pp. 578-585.
DOI
[8]
Honigman B., Lee J., Rothschild J., Light P., Pulling R. M., Yu T., and Bates D. W., Using computerized data to identify adverse drug events in outpatients, J. Am. Med. Inform. Assoc., vol. 8, no. 3, pp. 254-266, 2001.
DOI Google Scholar
[9]
Duan L., Khoshneshin M., Street W. N., and Liu M., Adverse drug effect detection, IEEE Journal of Biomedical & Health Informatics, vol. 17, no. 2, pp. 305-311, 2013.
DOI Google Scholar
[10]
Bates D. W., Evans R. S., Murff H., Stetson P. D., Pizziferri L., and Hripcsak G., Detecting adverse events using information technology, Journal of the American Medical Informatics Association, vol. 10, no. 2, pp. 115-128, 2003.
DOI Google Scholar
[11]
Han Z. J. and Zhang Y. J., Spark: A big data processing platform based on memory computing, in Proc. 7th Int. Symposium on Parallel Architectures, Algorithms and Programming, Nanjing, China, 2015, pp. 172-176.
DOI
[12]
Solovyev A., Mikheev M., Zhou L. M., Dutta-Moscato J., Ziraldo C., An G., Vodovotz Y., and Mi Q., SPARK: A framework for multi-scale agent-based biomedical modeling, International Journal of Agent Technologies & Systems, vol. 2, no. 3, pp. 18-30, 2010.
DOI Google Scholar
[13]
Cao P., Optimization and implementation of clustering algorithm based on Spark platform, (in Chinese), master degree dissertation, Beijing Jiaotong University, Beijing, China, 2016.
[14]
Zhang A., Insider of Spark Technology, (in Chinese). Beijing, China: Mechanical Industry Press, 2015.
[15]
Fu J., Sun J. W., and Wang K. Y., SPARK—A big data processing platform for machine learning, in Proc. 2016 International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration, Wuhan, China, 2017, pp. 48-51.
DOI
[16]
Brewer T. and Colditz G. A., Postmarketing surveillance and adverse drug reactions: Current perspectives and future needs, JAMA, vol. 218, no. 9, pp. 824-829, 1999.
DOI Google Scholar
[17]
Lansner A. and Ekeberg Ö., A one-layer feedback artificial neural network with a Bayesian learning rule, International Journal of Neural Systems, vol. 1, no. 1, pp. 77-87, 1989.
DOI Google Scholar
[18]
Karau H., Konwinski A., Wendell P., and Zaharia M., Learning Spark: Lightning-Fast Data Analysis, (in Chinese). Beijing, China: People’s Posts and Telecommunications Press, 2015.
[19]
Li W. Y., Research on apache spark for big data processing, (in Chinese), Modern Computer, no. 8, pp. 55-60, 2015.
Google Scholar
[20]
Xie S. L., Research and application of distributed ETL based on spark, (in Chinese), master degree dissertation, Donghua University, Shanghai, China, 2017.
[21]
FDA adverse event reporting system (FAERS): Latest quarterly data files, https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm082193.htm, 2017.
[22]
Wu Z. F., Zhang T., and Xiao Y., Improvement and parallel implementation of K-means clustering algorithm based on the Spark platform, (in Chinese), China Internet, no. 1, pp. 44-50, 2016.
Google Scholar
[23]
Wang P. F. and Li L., Research on multi-pattern matching algorithms based on Aho-Corasick algorithm, (in Chinese), Application Research of Computers, vol. 28, no. 4, pp. 1251–1253&1259, 2011.
Google Scholar
[24]
Aho A. V. and Corasick M. J., Efficient string matching: An aid to bibliographic search, Communications of the ACM, vol. 18, no. 6, pp. 333-340, 1975.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 16 August 2017
Accepted: 10 January 2018
Published: 31 December 2018
Issue date: April 2019

Copyright

© The author(s) 2019

Acknowledgements

This work was supported in part by the Science and Technology Innovation Action Plan Project of Science and Technology Commission of Shanghai Municipality (No. 18511102703) and the Scientific Research Plan Project of Science and Technology Commission of Shanghai Municipality (No. 16JC1400803).

Rights and permissions

Return