Output Type Guided Random Test Case Generation for String Validation Routines

String validation routines have been widely used in many real-world applications, such as email validation and postcode validation. String test cases are adopted to test these validation routines, to identify potential defects and security risks. Random Testing (RT) is a well-known testing approach to randomly generate string test cases from the input domain (i.e., the set of all possible test inputs), which is simple to implement at a low cost. However, its testing effectiveness may be unsatisfactory for string validation routines. The main reason for this is that RT may have a high probability to generate invalid rather than valid string test cases, due to its randomness property. This research proposes a new RT approach based on the output types (i.e., valid and invalid strings) for string validation routines, namely Output-type-guided Random Testing (RTO), which attempts to randomly generate both valid and invalid string test cases with a certain probability. This research performed an empirical study involving several real-world string validation routines collected from ten Java open-source projects, to investigate and compare testing performances of RT-O against the previous two widely-used RT methods. The results show that the generated string test cases by RT-O outperform test cases generated by other RT methods.