Journal Home > Volume 27 , Issue 1

Integer overflow is a common vulnerability in Ethereum Smart Contracts (ESCs) and often causes huge economic losses. Smart contracts cannot be changed once it is deployed on the blockchain and thus demand further testing. Mutation testing is a fault-based testing method that can effectively improve the sufficiency of a test for smart contracts. However, existing methods cannot efficiently perform mutation testing specifically for integer overflow in ESCs. Therefore, by analyzing integer overflow in ESCs, we propose five special mutation operators to address such vulnerability in terms of detecting sufficiency in ESC testing. An empirical study on 40 open-source ESCs is conducted to evaluate the effectiveness of the proposed mutation operators. Results show that (1) our proposed mutation operators can reproduce all 179 integer overflow vulnerabilities in 40 smart contracts, and the generated mutants have high compilation pass rate and integer overflow vulnerability generation rate; moreover, (2) the generated mutants can find the shortcomings of existing testing methods for integer overflow vulnerability, thereby providing effective support to improve the sufficiency of the test.


menu
Abstract
Full text
Outline
About this article

Mutation Testing for Integer Overflow in Ethereum Smart Contracts

Show Author's information Jinlei SunSong Huang( )Changyou Zheng( )Tingyong WangCheng ZongZhanwei Hui
Command & Control Engineering College, Army Engineering University of PLA, Nanjing 210000, China
Institute of Evaluation and Assessment Research, Academy of Military Science, Beijing 100091, China

Abstract

Integer overflow is a common vulnerability in Ethereum Smart Contracts (ESCs) and often causes huge economic losses. Smart contracts cannot be changed once it is deployed on the blockchain and thus demand further testing. Mutation testing is a fault-based testing method that can effectively improve the sufficiency of a test for smart contracts. However, existing methods cannot efficiently perform mutation testing specifically for integer overflow in ESCs. Therefore, by analyzing integer overflow in ESCs, we propose five special mutation operators to address such vulnerability in terms of detecting sufficiency in ESC testing. An empirical study on 40 open-source ESCs is conducted to evaluate the effectiveness of the proposed mutation operators. Results show that (1) our proposed mutation operators can reproduce all 179 integer overflow vulnerabilities in 40 smart contracts, and the generated mutants have high compilation pass rate and integer overflow vulnerability generation rate; moreover, (2) the generated mutants can find the shortcomings of existing testing methods for integer overflow vulnerability, thereby providing effective support to improve the sufficiency of the test.

Keywords: blockchain, Ethereum Smart Contracts (ESCs), integer overflow, mutation testing

References(49)

[1]
S. Nakamoto, Bitcoin: A peer-to-peer electronic cash system, https://bitcoin.org/en/bitcoin-paper, 2008.
[2]
G. Wood, Ethereum: A secure decentralised generalised transaction ledger, http://gavwood.com/Paper.pdf, 2014.
[3]
[4]
I. Nikolic A. Kolluri, I. Sergey, P. Saxena, and A. Hobor, Finding the greedy, prodigal, and suicidal contracts at scale, arXiv preprint arXiv: 1802.06038v2, 2018.
[5]
D. Siegel, Understanding the DAO attack, https://www.coindesk.com/understanding-dao-hackjournalists, 2016.
[6]
Parity Technologies, A postmortem on the parity multi-sig library self-destruct, https://www.parity.io/a-postmortemon-the-parity-multi-sig-library-self-destruct/, 2017.
[7]
[8]
S. Kalra, S. Goel, M. Dhawan, and S. Sharma, ZEUS: Analyzing safety of smart contracts, in Network and Distributed System Security Symp., San Diego, CA, USA, .
DOI
[9]
H. Wu, X. Wang, J. Xu, W. Zou, L. Zhang, and Z. Chen, Mutation testing for ethereum smart contract, arXiv preprint arXiv: 1908.03707, 2019.
[10]
[11]
N. Szabo, Smart contracts: Building blocks for digital markets, https://kameir.com/smart-contracts/, 1996.
[12]
Y. Jia and M. Harman, An analysis and survey of the development of mutation testing, IEEE Trans. Software Eng., vol. 37, no. 5, pp. 649-678, 2011.
[13]
R. A. DeMillo, R. J. Lipton, and F. G. Sayward, Hints on test data selection: Help for the practicing programmer, Computer, vol. 11, no. 4, pp. 34-41, 1978.
[14]
A. J. Offutt, Investigations of the software testing coupling effect, ACM Trans. Software Eng. Methodol., vol. 1, no. 1, pp. 3-18, 1992.
[15]
R. Ma, S. Ren, K. Ma, C. Hu, and J. Xue, Semi-valid fuzz testing case generation for stateful network protocol, Tsinghua Science and Technology, vol. 22, no. 5, pp. 458-468, 2017.
[16]
L. M. Zhang, T. Xie, L. Zhang, N. Tillmann, J. De Halleux, and H. Mei, Test generation via dynamic symbolic execution for mutation testing, in Proc. 2010 IEEE Int. Conf. Software Maintenance, Timisoara, Romania, 2010.
[17]
M. C. Sánchez, J. M. C. de Gea, J. L. Fernández-Alemán, J. Garceran, and A. T. Sánchez. Software vulnerabilities overview: A descriptive study, Tsinghua Science and Technology, vol. 25, no. 2, pp. 270-280, 2020.
[18]
W. E. Wong and A. P. Mathur, Reducing the cost of mutation testing: An empirical study, J. Syst. Softw., vol. 31, no. 3, pp. 185-196, 1995.
[19]
P. G. Frankl, S. N. Weiss, and C. Hu, All-uses vs mutation testing: An experimental comparison of effectiveness, J. Syst. Softw., vol. 38, no. 3, pp. 235-253, 1997.
[20]
M. Polo, M. Piattini, and I. García-Rodríguez, Decreasing the cost of mutation testing with second-order mutants, Softw. Test. Verif. Reliab., vol. 19, no. 2, pp. 111-131, 2009.
[21]
H. Coles, T. Laurent, C. Henard, M. Papadakis, and A. Ventresque, PIT: A practical mutation testing tool for Java, in Proc. 25th Int. Symp, Saarbrücken, Germany, 2016.
[22]
A. Derezinska and A. Szustek, Object-oriented testing capabilities and performance evaluation of the C# mutation system, in Proc. 4th IFIP TC 2 Central and East European Conf. Software Engineering Techniques, Krakow, Poland, 2012, pp. 229-242.
[23]
P. Delgado-Pérez, I. Medina-Bulo, F. Palomo-Lozano, A. García-Domínguez, and J. J. Domínguez-Jiménez, Assessment of class mutation operators for C++ with the MuCPP mutation system, Inf. Softw. Technol., vol. 81, pp. 169-184, 2017.
[24]
Y. S. Ma, M. J. Harrold, and Y. R. Kwon, Evaluation of mutation testing for object-oriented programs, presented at 28th Int. Conf. Software Engineering, Shanghai, China, 2006.
[25]
S. W. Kim, J. A. Clark, and J. A. Mcdermid, Assessing test set adequacy for object-oriented programs using class mutation, presented at Symp. Class Mutation, York, England, 2016.
[26]
S. W. Kim, J. A. Clark, and J. A. Mcdermid, Class mutation: mutation testing for object-oriented programs, in Proc. Conf. Object-Oriented Software Systems, Erfurt, Germany, 2000.
[27]
K. Claessen and J. Hughes, QuickCheck: A lightweight tool for random testing of Haskell programs, ACM SIGPLAN Not., vol. 46, no. 4, pp. 268-279, 2000.
[28]
D. Le, M. A. Alipour, R. Gopinath, and A. Groce, MuCheck: An extensible tool for mutation testing of haskell programs, in Proc. 2014 Int. Symp. Software Testing and Analysis, San Jose, CA, USA, 2014.
[29]
L. Deng, N. Mirzaei, P. Ammann, and J. Offutt, Towards mutation analysis of Android apps, in Proc. 8th Int. Conf. Software Testing Verification and Validation Workshops (ICSTW), Graz, Austria, 2015.
[30]
K. Moran, M. Tufano, C. Bernal-Cárdenas, M. Linares-Vásquez, G. Bavota, C. Vendome, M. D. Penta, and D. Poshyvanyk, MDroid+: A mutation testing framework for android, in Proc. 2018 IEEE/ACM 40th Int. Conf. Software Engineering: Companion (ICSE-Companion), Gothenburg, Sweden, 2018.
[31]
S. Mirshokraie, A. Mesbah, and K. Pattabiraman, Guided mutation testing for JavaScript web applications, IEEE Trans. Softw. Eng., vol. 41, no. 5, pp. 429-444, 2015.
[32]
J. Chen, H. Wang, D. Towey, C. Mao, R. Huang, and Y. Zhan, Worst-input mutation approach to web services vulnerability testing based on SOAP messages, Tsinghua Science and Technology, vol. 19, no. 5, pp. 429-441, 2014.
[33]
Z. X. Li, H. R. Wu, J. H. Xu, X. Y. Wang, L. M. Zhang, and Z. Y. Chen, MuSC: A tool for mutation testing of ethereum smart contract, in Proc. 34th IEEE/ACM Int. Conf. Automated Software Engineering (ASE), San Diego, CA, USA, 2019.
[34]
P. Hartel and R. Schumi, Mutation testing of smart contracts at scale, arXiv preprint arXiv: 1909.12563, 2019.
[35]
E. Andesta, F. Faghih, and M. Fooladgar, Testing smart contracts gets smarter, arXiv preprint arXiv: 1912.04780, 2019.
[36]
L. Luu, D. H. Chu, H. Olickel, P. Saxena, and A. Hobor, Making smart contracts smarter, in Proc. 2016 ACM SIGSAC Conf. Computer and Communications Security, Vienna, Austria, 2016.
[37]
C. F. Torres, J. Schütte, and R. State, Osiris: Hunting for integer bugs in ethereum smart contracts, in Proc. 34th Ann. Computer Security Applications Conf. (ACSAC), San Juan, PR, USA, 2018.
[38]
K. Bhargavan, N. Swamy, S. Zanella-Béguelin, and A. Delignat-Lavaud, Formal verification of smart contracts: Short paper, in Proc. 34th Ann. Computer Security Applications Conf. (ACSAC’18), San Juan, PR, USA, 2016.
[39]
Y. Hirai, Formal verfication of Deed contract in Ethereum name service, https://yoichihirai.com/deed.pdf, 2016.
[40]
P. Tsankov, A. Dan, D. D. Cohen, A. Gervais, F. Buenzli, and M. Vechev, Securify: Practical security analysis of smart contracts, arXiv preprint arXiv: 1806.01143, 2018.
[41]
S. Tikhomirov, E. Voskresenskaya, I. Ivanitskiy, R. Takhaviev, E. Marchenko, and Y. Alexandrov, SmartCheck: Static analysis of ethereum smart contracts, in Proc. 2018 IEEE/ACM 1st Int. Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), Gothenburg, Sweden, 2018.
[42]
P. C. Zhang, F. Xiao, and X. P. Luo, SolidityCheck: Quickly detecting smart contract problems through regular expressions, arXiv preprint arXiv: 1911.09425v1, 2019.
[43]
J. Feist, G. Grieco, and A. Groce, Slither: A static analysis framework for smart contracts, in Proc. 2019 IEEE/ACM 2nd Int. Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), Montreal, Canada, 2019.
[44]
T. Durieux, J. F. Ferreira, R. Abreu, and P. Cruz, Empirical review of automated analysis tools on 47587 Ethereum Smart Contracts, arXiv preprint arXiv: 1910.10601, 2019.
[45]
B. Jiang, Y. Liu, and W. K. Chan, ContractFuzzer: Fuzzing smart contracts for vulnerability detection, presented at 2018 33rd IEEE/ACM Int. Conf. Automated Software Engineering (ASE), Montpellier, France, 2018.
[46]
H. J. Wang, Y. Li, S. W. Lin, C. Artho, L. Ma, and Y. Liu, Oracle-supported dynamic exploit generation for smart contracts, arXiv preprint arXiv: 1909.06605, 2019.
[47]
C. Liu, H. Liu, Z. Cao, Z. Chen, B. D. Chen, and B. Roscoe, ReGuard: Finding reentrancy bugs in smart contracts, in Proc. ACM/IEEE 40th Int. Conf. Software Engineering, Gothenburg, Sweden, 2018, pp. 65-68.
[48]
[49]
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 08 August 2020
Accepted: 09 September 2020
Published: 17 August 2021
Issue date: February 2022

Copyright

© The author(s) 2022

Acknowledgements

The project was supported by National Key R&D Program of China (No. 2018YFB1403400), the National Natural Science Foundation of China (No. 61702544), Natural Science Foundation of Jiangsu Province, China (Nos. BK20160769 and BK20141072), and China Postdoctoral Science Foundation (No. 2016M603031).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return