Form Tester: Website Test Automation Using Form-Oriented Models

Abstract

Whilst Model Based Testing (MBT) is an improvement over manual test specification, the leap from manual testing to MBT can be hard. Current MBT tools only allow development of models from scratch, without reuse of existing manual test specifications. Conversely, current MBT tools do not facilitate partial automation of manual test cases, e.g. to automate only the test initialization procedure. Finally, existing MBT tools do not identify areas of the system that are not testable due to defects, which may in turn conceal other defects. We propose a new MBT approach that addresses these shortcomings. The approach uses a form-oriented model that clearly represents the actions, pages and fields of a web application. Instead of having to specify a test model from scratch, existing manual test cases can be used without modification as part of a test model specification. This means that developers can move from manual testing to MBT without any additional cost. Furthermore, the approach allows developers to use test models to partially automate manual tests. A test model can be used to automate the initialization step of manual test cases, hence making it easier to specify manual tests. Finally, the proposed approach can identify areas of the application that are unreachable due to defects using an online testing strategy. This provides an overview of all testable parts that have been covered, as well as areas where additional defects may be concealed. The approach was implemented in a novel MBT tool called FormTester, and evaluated with real web applications. One small and one mid-sized application were tested using manually specified tests; one large, complex application was automated using the MBT tool. The tool's performance gains over manual specification were evaluated. Manual test cases were used to specify the model, and their test initializations were generated automatically. This improved time spent developing test cases (Minutes/Test) by about 30%. The same types and similar numbers of defects were discovered using the tool as manual specification, indicating a similar effectiveness in defect detection.