Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Similarity-based Web Element Localization for Robust Test Automation
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0002-8569-2290
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-7526-3727
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0002-5179-4205
University Genoa, Italy.
Show others and affiliations
2023 (English)In: ACM Transactions on Software Engineering and Methodology, ISSN 1049-331X, E-ISSN 1557-7392, Vol. 32, no 3, article id 75Article in journal (Refereed) Published
Abstract [en]

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This article proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be 4 milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2023. Vol. 32, no 3, article id 75
Keywords [en]
GUI testing, test automation, test case robustness, web element locators, XPath locators
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-25065DOI: 10.1145/3571855ISI: 001002573400020OAI: oai:DiVA.org:bth-25065DiVA, id: diva2:1777888
Part of project
SERT- Software Engineering ReThought, Knowledge Foundation
Funder
Knowledge Foundation, 20180010Swedish Research Council, 2015-04913Swedish Research Council, 2020-05272Available from: 2023-06-30 Created: 2023-06-30 Last updated: 2023-11-22Bibliographically approved
In thesis
1. On overcoming challenges with GUI-based test automation
Open this publication in new window or tab >>On overcoming challenges with GUI-based test automation
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Background: Automated testing is widely used in modern software development to check if the software, including its graphical user interface (GUI), meets the expectations in terms of quality and functionality. GUI-based test automation, like other automation, aims to save time and money compared to manual testing without reducing the software quality. While automation has successfully reduced costs for other types of testing (e.g., unit-or integration tests), GUI-based testing has faced technical challenges, some of which have lingered for over a decade. 

Objective: This thesis work aims to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate some of the main challenges. One such challenge is to reliably identify GUI elements during test execution to prevent unnecessary repairs. Another problem is the demand for test automation and programming skills when designing stable automated tests at scale. 

Method: We conducted several studies by adopting a multi-methodological approach. First, we performed a systematic literature review to identify the main challenges in GUI-based test automation, followed by multiple studies that propose and evaluate novel approaches to mitigate the main challenges. 

Results: Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e. most reported) on a timeline and classified them as essential or accidental. This classification is valuable since future research can focus on the main challenges that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several approaches that explore novel concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges. Testing an application through an augmented layer (Augmented Testing) can reduce the demand for test automation and programming skills and mitigate the challenges of creating and maintaining model based tests. Our proposed approach for locating web elements (Similo) can increase the robustness of automated test execution. 

Conclusion: Our results provide alternative approaches and concepts that can mitigate some of the main accidental challenges in GUI-based test automation. With a more robust test execution and tool support for test modeling, we can help reduce the manual labor spent on creating and maintaining automated GUI-based tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2024. p. 215
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 2
Keywords
GUI Testing, Test Automation, Augmented Testing, Test Case Robustness, Web Element Locators, Large Language Models
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-25638 (URN)978-91-7295-473-1 (ISBN)
Public defence
2024-02-06, J1630, Campus Karlskrona, 13:00 (English)
Opponent
Supervisors
Available from: 2023-11-28 Created: 2023-11-22 Last updated: 2024-02-13Bibliographically approved

Open Access in DiVA

fulltext(2377 kB)115 downloads
File information
File name FULLTEXT01.pdfFile size 2377 kBChecksum SHA-512
8f060d63070b7bc372e7e9159576d757b70b091ac395bdabf21bd6c79c70fc0a1466a81c6b5ce46574c1298e7817db8cdff6099d8187a44c8a254e662f02b822
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Authority records

Nass, MichelAlégroth, EmilFeldt, Robert

Search in DiVA

By author/editor
Nass, MichelAlégroth, EmilFeldt, Robert
By organisation
Department of Software Engineering
In the same journal
ACM Transactions on Software Engineering and Methodology
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 116 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 299 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf