Context: Automated acceptance tests have become a core practice of agile software development (e.g. Extreme Programming). These tests are closely tied to requirements specifications and these tests provide a mechanism for continuous validation of software requirements. Software reuse has evolved with the introduction of each new reusable artefact (e.g., reuse of code, reuse of frameworks, tools etc.). In this study, we have investigated the reusability of automated acceptance tests keeping in view their close association with textual requirements.

Objective: As automated acceptance tests are closely related to software requirements, we have used existing research in software engineering to identify reusability related characteristics of software requirements and used these characteristics for automated acceptance tests.This study attempts to address the following aspects: (i) what important reuse characteristics should be considered when measuring reusability of automated acceptance tests? (ii) how reusability can be measured in automated acceptance tests?, and (iii) how cost avoided through reuse of automated acceptance tests can be calculated?

Method: We have used a combination of research methods to answer different aspects of our study. We started by identifying reusability related characteristics of software requirements, with help of systematic literature review. Later, we tried to identify the reusability related characteristics of defect reports and the process is documented using an experience report. After identifying the characteristics from the previous two studies, we used these characteristics on two case-studies conducted on Behaviour driven development test cases (i.e., acceptance tests of textual nature). We proposed two approaches that can identify the reuse potential of automated acceptance tests and evaluated these approaches in the industry. Later, to calculate the cost avoided through reuse, we proposed and evaluated a method that is applicable to any reusable artifact.

Results: The results from the systematic literature review shows that text-based requirements reuse approaches are most commonly used in the industry. Structuring these text-based requirements and identifying the reusable requirements by matching are the two commonly used methods for enabling requirements to reuse. The results from the second study, industrial experience report, indicates that defect reports can be formulated in template and defect triage meeting can be used to identify important test-cases related to defect reports. The results from these two studies, text-based requirements reuse approaches and template based defect reports, were included when identifying approaches to measure reuse potential of BDD test-cases. The two proposed approaches, Normalised Compression Distance (NCD) and Similarity Ratio, for measuring reuse potential were evaluated in the industry. The evaluation indicated that Similarity ratio approach performed better than the NCD approach, however, the results from both approaches were comparable with the results gathered with the help of expert analysis. The cost related aspects of reusable acceptance tests were addressed and evaluated using a method that calculates the cost-avoidance through reuse. The industrial evaluation of the method and guidelines show that the method is an artifact independent method. 

Conclusions: The evidence from this study shows that the automated acceptance tests are reusable, similar to text-based software requirements and their reuse potential can be calculated as well. The industrial evaluation of the three studies (i.e. approaches to measure reuse potential, calculation of cost avoidance and defect reports in triage meetings) shows that the overall results are applicable to the industry. However, further work is required to evaluate the reuse potential of automated acceptance tests in different contexts. 

Context: Automated acceptance testing has become a relevant practice of agile software development (e.g., Extreme Programming). Automated acceptance tests (AATs) are closely tied to requirements and provide a mechanism for continuous validation of requirements as system-level tests. Studies have shown that these tests are costly to develop, maintain, and reuse.

Objectives: This thesis examines and supports the systematic reuse of automated acceptance tests by providing approaches to search, identity, adapt reusable test cases while considering the reuse costs. Additionally, the maintainability and usage of automated acceptance tests in largescale software projects are studied.

Method: We used various research methods to investigate development, reuse, and maintaining the automated acceptance tests. Workshops and interviews of practitioners were used to identify the benefits and challenges of automated acceptance tests. Next, method engineering was used to construct a systematic reuse process for automated acceptance tests. Qualitative feedback collected using a survey, and industrial demonstration examined the reuse process’s performance expectancy, effort expectancy, and necessary facilitating conditions. Two systematic literature reviews are used to identify techniques to adapt tests for future reuse opportunities and calculate the reuse costs of automated acceptance tests. Later, we developed and evaluated an approach for refactoring behavior-driven development-based automated acceptance tests using action research.

Results: A cost-aware systematic reuse process was constructed to support the reuse of automated acceptance tests containing eleven activities. For each activity, guidelines on expected input, expected output, the actors performing the activity, and techniques (automated using scripts) are suggested and evaluated. The techniques involve approaches to support development for reuse and methods to calculate the costs of reusing automated acceptance tests. The industrial evaluation of the reuse process and the techniques showed its usefulness and relevance for the industry. Furthermore, seven challenges (i.e., the scale of the software projects, ownership, lack of competence, cost benefits, specification of behaviors in large-scale projects, difficulty writing system-level test-cases, and versioning control of behaviors) and five benefits (i.e., understanding of a business aspect of requirements, improved quality of requirements, a guide to system-level use-cases, reuse of artifacts in large-scale projects, and help for test organization.) of using automated acceptance tests in large-scale projects are identified with software practitioners’ help. Later, we proposed a semiautomated four-step approach for pre-processing, measuring, ranking, and identifying refactoring candidates. The approach and the two proposed measures were successfully evaluated using two industrial projects. It was noted that similarity measures could support the maintenance of the specification base using refactoring.

Conclusions: The studies show that automated acceptance tests are reusable and can be refactored using our proposed approach. The evaluation shows that the results apply to the software industry in the evaluated context. However, further work is required to evaluate the reuse process in different contexts.