Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Augmented testing: Industry feedback to shape a new testing technology
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
Chalmers, SWE.
2019 (English)In: Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019, Institute of Electrical and Electronics Engineers Inc. , 2019, p. 176-183Conference paper, Published paper (Refereed)
Abstract [en]

Manual testing is the most commonly used approach in the industry today for acceptance-and system-testing of software applications. Test automation has been suggested to address drawbacks with manual testing but both test automation and manual testing have several challenges that limit their return of investment for system-and acceptance-test automation. Hence, there is still an industrial need for another approach to testing that can mitigate the challenges associated with system-and acceptance-testing and make it more efficient and cost effective for the industry. In this paper we present a novel technique we refer to as Augmented Testing (AT). AT is defined as testing through a visual layer between the tester and the System Under Test (SUT) that superimposes information on top of the GUI. We created a prototype for AT and performed an industrial workshop study with 10 software developers to get their perceived benefits and drawbacks of AT. The benefits and drawbacks will be useful for further development of the technique and prototype for AT. The workshop study identified more benefits than drawbacks with AT. Two of the identified benefits were: 'Know what to test and what has been tested' and 'Less manual work'. Due to these results, we believe that AT is a promising technique that deserves more research since it may provide industry with new benefits that current techniques lack. © 2019 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2019. p. 176-183
Series
IEEE International Conference on Software Testing Verification and Validation Workshops
Keywords [en]
Augmented Testing, Industrial Workshop Study, System Testing, Test Automation, Application programs, Automation, Cost effectiveness, Software prototyping, Software testing, System theory, Verification, Acceptance testing, Return of investments, Software applications, Software developer, Testing technology, Acceptance tests
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-18591DOI: 10.1109/ICSTW.2019.00048ISI: 000477742600024Scopus ID: 2-s2.0-85068385060ISBN: 9781728108889 (print)OAI: oai:DiVA.org:bth-18591DiVA, id: diva2:1349336
Conference
12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Xi'an, China, 22 April 2019 through 27 April 2019
Part of project
SERT- Software Engineering ReThought, Knowledge FoundationAvailable from: 2019-09-09 Created: 2019-09-09 Last updated: 2023-11-22Bibliographically approved
In thesis
1. On overcoming challenges with GUI-based test automation
Open this publication in new window or tab >>On overcoming challenges with GUI-based test automation
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Background: Automated testing is widely used in modern software development to check if the software, including its graphical user interface (GUI), meets the expectations in terms of quality and functionality. GUI-based test automation, like other automation, aims to save time and money compared to manual testing without reducing the software quality. While automation has successfully reduced costs for other types of testing (e.g., unit-or integration tests), GUI-based testing has faced technical challenges, some of which have lingered for over a decade. 

Objective: This thesis work aims to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate some of the main challenges. One such challenge is to reliably identify GUI elements during test execution to prevent unnecessary repairs. Another problem is the demand for test automation and programming skills when designing stable automated tests at scale. 

Method: We conducted several studies by adopting a multi-methodological approach. First, we performed a systematic literature review to identify the main challenges in GUI-based test automation, followed by multiple studies that propose and evaluate novel approaches to mitigate the main challenges. 

Results: Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e. most reported) on a timeline and classified them as essential or accidental. This classification is valuable since future research can focus on the main challenges that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several approaches that explore novel concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges. Testing an application through an augmented layer (Augmented Testing) can reduce the demand for test automation and programming skills and mitigate the challenges of creating and maintaining model based tests. Our proposed approach for locating web elements (Similo) can increase the robustness of automated test execution. 

Conclusion: Our results provide alternative approaches and concepts that can mitigate some of the main accidental challenges in GUI-based test automation. With a more robust test execution and tool support for test modeling, we can help reduce the manual labor spent on creating and maintaining automated GUI-based tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2024. p. 215
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 2
Keywords
GUI Testing, Test Automation, Augmented Testing, Test Case Robustness, Web Element Locators, Large Language Models
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-25638 (URN)978-91-7295-473-1 (ISBN)
Public defence
2024-02-06, J1630, Campus Karlskrona, 13:00 (English)
Opponent
Supervisors
Available from: 2023-11-28 Created: 2023-11-22 Last updated: 2024-02-13Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Alégroth, Emil

Search in DiVA

By author/editor
Nass, MichelAlégroth, Emil
By organisation
Department of Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 409 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf