Background: Modern software systems are large and complex, requiring collaboration among developers with diverse skills to manage this complexity. Code review is an essential collaborative software engineering practice in which changes are discussed before they are integrated into the codebase, enhancing code quality and promoting knowledge sharing. GUI-based testing is a technique that verifies and validates a system’s behavior through its GUI by simulating user interactions. Like production code, it requires collaboration, as tests are created, reviewed, and maintained alongside production code. Code review practices for tests differ from those for production code. Omitting reviews of tests can lower their quality and increase maintenance costs. However, practices for reviewing GUI-based tests are not well understood, as academic literature mainly focuses on production code and low-level tests.

Objective: We aim to advance the understanding and practice of code review of GUI-based tests by (a) identifying code review guidelines; (b) investigating the specific practices, challenges, and information needs; (c) finding empirical evidence supporting the proposed guidelines; and (d) providing an outlook on how code review may evolve in the future.

Methods: First, we conduct a literature review of white and gray literature to identify guidelines for source and test code, and synthesize them for GUI-based tests. Next, we perform qualitative interviews with software testing professionals to identify practices, challenges, and information needs when reviewing GUI-based tests. To find empirical evidence for the proposed guidelines, we mine open-source software repositories. Finally, we conduct a questionnaire survey to gather practitioners’ expectations about the future importance of code reviews.

Results: We synthesized 33 guidelines for GUI-based tests from literature sources. In analyzing code review comments from open-source repositories, we found empirical evidence supporting 25 out of the 33 proposed guidelines. Practitioners acknowledge the importance of code reviews, but lack defined practices for reviewing GUI-based tests. We identified four practices, six challenges, and four information needs related to reviewing GUI-based tests. The survey results indicate that code review will remain an essential practice with an anticipated increase in code review activities, including those for GUI-based tests.

Conclusion: This thesis advances the understanding and practice of code review for GUI-based tests to improve both review effectiveness and the quality of the tests underreview. We present a set of empirically grounded guidelines derived from literature and refined through the analysis of code review comments of open-source repositories. Our research investigates current practices for reviewing GUI-based tests, highlighting the specific challenges and information needs that distinguish these reviews from those for production code. Finally, we highlight the relevance of our research for the future.