Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
LLM-Based Labelling of Recorded Automated GUI-Based Test Cases
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0003-1432-5255
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-7526-3727
Synteda Ab, Gothenburg, Sweden.
2025 (English)In: 2025 IEEE Conference on Software Testing, Verification and Validation, ICST 2025 / [ed] Fasolino A.R., Panichella S., Aleti A., Mesbah A., Institute of Electrical and Electronics Engineers (IEEE), 2025, p. 453-463Conference paper, Published paper (Refereed)
Abstract [en]

Graphical User Interface (GUI) based testing is a commonly used practice in industry. Although valuable and, in many cases, necessary, it is associated with challenges such as high cost and requirements on both technical and domain expertise. Augmented testing, a novel approach to GUI test automation, aims to mitigate these challenges by allowing users to record and render test cases and test data directly on the GUI of the system under test (SUT). In this context, Scout is an augmented testing tool that captures system states and transitions during manual interaction with the SUT, storing them in a test model that is visually represented in the form of state trees and reports. While this representation provides basic overview of a test suite, e.g. about its size and number of scenarios, it is limited in terms of analysis depth, interpretability, and reproducibility. In particular, without human state labeling, it is challenging to produce meaningful and easily understandable test reports. To address this limitation, we present a novel solution and a demonstrator, integrated into Scout, which leverages large language models (LLMs) to enrich the model-based test case representation by automatically labeling and describing states and describing transitions. We conducted two experiments to evaluate the impact of the solution. First, we compared LLM-enhanced reports with expert-generated reports using embedding distance evaluation metrics. Second, we assessed the usability and perceived value of the enhanced reports through an industrial survey. The results of the study indicate that the plugin can improve readability, actionability, and interpretability of test reports. This work contributes to the automation of GUI testing by reducing the need for manual intervention, e.g. labeling, and technical expertise, e.g. to understand test case models. Although the solution is studied in the context of augmented testing, we argue for the solution's generalizability to related test automation techniques. In addition, we argue that this approach enables actionable insights and lays the groundwork for further research into autonomous testing based on Generative AI. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025. p. 453-463
Keywords [en]
High costs, Interpretability, Labelings, Language model, Model-based OPC, Systems under tests, Technical expertise, Test Automation, Test case, Test reports, Graphical user interfaces
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-28173DOI: 10.1109/ICST62969.2025.10988984ISI: 001506893900040Scopus ID: 2-s2.0-105007522870ISBN: 9798331508142 (print)OAI: oai:DiVA.org:bth-28173DiVA, id: diva2:1974502
Conference
18th IEEE Conference on Software Testing, Verification and Validation, ICST 2025, Naples, April 31-4, 2025
Part of project
SERT- Software Engineering ReThought, Knowledge FoundationT.A.R.G.E.T. – Testing with AI Reinforced GUI Embedded Technology, Vinnova
Funder
Knowledge Foundation, 20180010Vinnova, 2024-00242Available from: 2025-06-23 Created: 2025-06-23 Last updated: 2025-09-30Bibliographically approved

Open Access in DiVA

fulltext(1419 kB)113 downloads
File information
File name FULLTEXT01.pdfFile size 1419 kBChecksum SHA-512
505a58ceeee103c80c253265041a1582fcd47995e1a8e09d36c16555a834e973f9b583f009538e1ae0d3367cb7679efb229d9a3ba2b60b1949b253f05713adb5
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Buarque Franzosi, DiogoAlégroth, Emil

Search in DiVA

By author/editor
Buarque Franzosi, DiogoAlégroth, Emil
By organisation
Department of Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 113 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 314 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf