Context:Contemporary software development is a socio-technical activity requiring extensive collaboration among individuals with diverse expertise.

Software testing is an integral part of software development that also depends on various expertise.

GUI-based testing allows assessing a system’s GUI and its behavior through its graphical user interface.

Collaborative practices in software development, like code reviews, not only improve software quality but also promote knowledge exchange within teams.

Similar benefits could be extended to other areas of software engineering, such as GUI-based testing.

However, collaborative practices for GUI-based testing necessitate a unique approach since general software development practices, perceivably, can not be directly transferred to software testing.

Goal:This thesis contributes towards a tool-supported approach enabling collaborative GUI-based testing.

Our distinct goals are (1) to identify processes and guidelines to enable collaboration on GUI-based testing artifacts and (2) to operationalize tool support to aid this collaboration.

Method:We conducted a systematic literature review identifying code review guidelines for GUI-based testing.

Further, we conducted a controlled experiment to assess the efficiency and potential usability issues of Augmented Testing.

Results:We provided guidelines for reviewing GUI-based testing artifacts, which aid contributors and reviewers during code reviews.

We further provide empirical evidence that Augmented Testing is not only an efficient approach to GUI-based testing but also usable for non-technical users, making it a promising subject for further research in collaborative GUI-based testing.

Conclusion:Code review guidelines aid collaboration through discussions, and a suitable testing approach can serve as a platform to operationalize collaboration.

Collaborative GUI-based testing has the potential to improve the efficiency and effectiveness of such testing.

Background: Modern software systems are large and complex, requiring collaboration among developers with diverse skills to manage this complexity. Code review is an essential collaborative software engineering practice in which changes are discussed before they are integrated into the codebase, enhancing code quality and promoting knowledge sharing. GUI-based testing is a technique that verifies and validates a system’s behavior through its GUI by simulating user interactions. Like production code, it requires collaboration, as tests are created, reviewed, and maintained alongside production code. Code review practices for tests differ from those for production code. Omitting reviews of tests can lower their quality and increase maintenance costs. However, practices for reviewing GUI-based tests are not well understood, as academic literature mainly focuses on production code and low-level tests.

Objective: We aim to advance the understanding and practice of code review of GUI-based tests by (a) identifying code review guidelines; (b) investigating the specific practices, challenges, and information needs; (c) finding empirical evidence supporting the proposed guidelines; and (d) providing an outlook on how code review may evolve in the future.

Methods: First, we conduct a literature review of white and gray literature to identify guidelines for source and test code, and synthesize them for GUI-based tests. Next, we perform qualitative interviews with software testing professionals to identify practices, challenges, and information needs when reviewing GUI-based tests. To find empirical evidence for the proposed guidelines, we mine open-source software repositories. Finally, we conduct a questionnaire survey to gather practitioners’ expectations about the future importance of code reviews.

Results: We synthesized 33 guidelines for GUI-based tests from literature sources. In analyzing code review comments from open-source repositories, we found empirical evidence supporting 25 out of the 33 proposed guidelines. Practitioners acknowledge the importance of code reviews, but lack defined practices for reviewing GUI-based tests. We identified four practices, six challenges, and four information needs related to reviewing GUI-based tests. The survey results indicate that code review will remain an essential practice with an anticipated increase in code review activities, including those for GUI-based tests.

Conclusion: This thesis advances the understanding and practice of code review for GUI-based tests to improve both review effectiveness and the quality of the tests underreview. We present a set of empirically grounded guidelines derived from literature and refined through the analysis of code review comments of open-source repositories. Our research investigates current practices for reviewing GUI-based tests, highlighting the specific challenges and information needs that distinguish these reviews from those for production code. Finally, we highlight the relevance of our research for the future.