Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Augmented Testing to support Manual GUI-based Regression Testing: An Empirical Study
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0002-2916-4020
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0003-3995-6125
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-7526-3727
2024 (English)In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 29, no 6, article id 140Article in journal (Refereed) Published
Abstract [en]

Context: Manual graphical user interface (GUI) software testing presents a substantial part of the overall practiced testing efforts, despite various research efforts to further increase test automation. Augmented Testing (AT), a novel approach for GUI testing, aims to aid manual GUI-based testing through a tool-supported approach where an intermediary visual layer is rendered between the system under test (SUT) and the tester, superimposing relevant test information.

Objective: The primary objective of this study is to gather empirical evidence regarding AT's efficiency compared to manual GUI-based regression testing. Existing studies involving testing approaches under the AT definition primarily focus on exploratory GUI testing, leaving a gap in the context of regression testing. As a secondary objective, we investigate AT's benefits, drawbacks, and usability issues when deployed with the demonstrator tool, Scout.

Method: We conducted an experiment involving 13 industry professionals, from six companies, comparing AT to manual GUI-based regression testing. These results were complemented by interviews and Bayesian data analysis (BDA) of the study's quantitative results.

Results: The results of the Bayesian data analysis revealed that the use of AT shortens test durations in 70% of the cases on average, concluding that AT is more efficient.When comparing the means of the total duration to perform all tests, AT reduced the test duration by 36% in total. Participant interviews highlighted nine benefits and eleven drawbacks of AT, while observations revealed four usability issues.

Conclusion: This study makes an empirical contribution to understanding Augmented Testing, a promising approach to improve the efficiency of GUI-based regression testing in practice. Furthermore, it underscores the importance of continual refinements of AT.

Place, publisher, year, edition, pages
Springer, 2024. Vol. 29, no 6, article id 140
Keywords [en]
GUI-based testing, GUI testing, Augmented Testing, manual teting, Bayesian data analysis
National Category
Software Engineering
Research subject
Systems Engineering
Identifiers
URN: urn:nbn:se:bth-25391DOI: 10.1007/s10664-024-10522-zISI: 001292331700002Scopus ID: 2-s2.0-85201391671OAI: oai:DiVA.org:bth-25391DiVA, id: diva2:1797942
Part of project
SERT- Software Engineering ReThought, Knowledge Foundation
Funder
Knowledge Foundation, 20180010Available from: 2023-09-18 Created: 2023-09-18 Last updated: 2024-08-30Bibliographically approved
In thesis
1. Towards Collaborative GUI-based Testing
Open this publication in new window or tab >>Towards Collaborative GUI-based Testing
2023 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Context:Contemporary software development is a socio-technical activity requiring extensive collaboration among individuals with diverse expertise.

Software testing is an integral part of software development that also depends on various expertise.

GUI-based testing allows assessing a system’s GUI and its behavior through its graphical user interface.

Collaborative practices in software development, like code reviews, not only improve software quality but also promote knowledge exchange within teams.

Similar benefits could be extended to other areas of software engineering, such as GUI-based testing.

However, collaborative practices for GUI-based testing necessitate a unique approach since general software development practices, perceivably, can not be directly transferred to software testing.

Goal:This thesis contributes towards a tool-supported approach enabling collaborative GUI-based testing.

Our distinct goals are (1) to identify processes and guidelines to enable collaboration on GUI-based testing artifacts and (2) to operationalize tool support to aid this collaboration.

Method:We conducted a systematic literature review identifying code review guidelines for GUI-based testing.

Further, we conducted a controlled experiment to assess the efficiency and potential usability issues of Augmented Testing.

Results:We provided guidelines for reviewing GUI-based testing artifacts, which aid contributors and reviewers during code reviews.

We further provide empirical evidence that Augmented Testing is not only an efficient approach to GUI-based testing but also usable for non-technical users, making it a promising subject for further research in collaborative GUI-based testing.

Conclusion:Code review guidelines aid collaboration through discussions, and a suitable testing approach can serve as a platform to operationalize collaboration.

Collaborative GUI-based testing has the potential to improve the efficiency and effectiveness of such testing.

Place, publisher, year, edition, pages
Karlskrona, Sweden: Blekinge Tekniska Högskola, 2023
Series
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 2023:10
Keywords
software testing, GUI testing, GUI-based testing, collaborative testing, code review
National Category
Software Engineering
Research subject
Systems Engineering
Identifiers
urn:nbn:se:bth-25392 (URN)978-91-7295-469-4 (ISBN)
Presentation
2023-11-01, J1630 + Zoom, BTH, Karlskrona, 13:00 (English)
Opponent
Supervisors
Available from: 2023-09-18 Created: 2023-09-18 Last updated: 2023-12-05Bibliographically approved

Open Access in DiVA

fulltext(1334 kB)32 downloads
File information
File name FULLTEXT01.pdfFile size 1334 kBChecksum SHA-512
44d9bb922c06e98465d804a890c58eace68131d357756f2157ad4a917eb68acb716d96e5820821effdbdcae9b957fe7c955f9876bb20a146893daa08cb533370
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Bauer, AndreasFrattini, JulianAlégroth, Emil

Search in DiVA

By author/editor
Bauer, AndreasFrattini, JulianAlégroth, Emil
By organisation
Department of Software Engineering
In the same journal
Empirical Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 32 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 311 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf