Change search
ReferencesLink to record
Permanent link

Direct link
Maintenance of automated test suites in industry: An empirical study on Visual GUI Testing
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
2016 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 73, 66-80 p.Article in journal (Refereed) PublishedText
Abstract [en]

Context: Verification and validation (V&V) activities make up 20-50% of the total development costs of a software system in practice. Test automation is proposed to lower these V&V costs but available research only provides limited empirical data from industrial practice about the maintenance costs of automated tests and what factors affect these costs. In particular, these costs and factors are unknown for automated GUI-based testing. Objective: This paper addresses this lack of knowledge through analysis of the costs and factors associated with the maintenance of automated GUI-based tests in industrial practice. Method: An empirical study at two companies, Siemens and Saab, is reported where interviews about, and empirical work with, Visual GUI Testing is performed to acquire data about the technique's maintenance costs and feasibility. Results: 13 factors are observed that affect maintenance, e.g. tester knowledge/experience and test case complexity. Further, statistical analysis shows that developing new test scripts is costlier than maintenance but also that frequent maintenance is less costly than infrequent, big bang maintenance. In addition a cost model, based on previous work, is presented that estimates the time to positive return on investment (ROI) of test automation compared to manual testing. Conclusions: It is concluded that test automation can lower overall software development costs of a project while also having positive effects on software quality. However, maintenance costs can still be considerable and the less time a company currently spends on manual testing, the more time is required before positive, economic, ROI is reached after automation. (C) 2016 Elsevier B.V. All rights reserved.

Place, publisher, year, edition, pages
2016. Vol. 73, 66-80 p.
Keyword [en]
Visual GUI Testing; Maintenance; Return on investment; Empirical; Industrial
National Category
Software Engineering
URN: urn:nbn:se:bth-11846DOI: 10.1016/j.infsof.2016.01.012ISI: 000373537400006OAI: diva2:925547
Available from: 2016-05-02 Created: 2016-05-02 Last updated: 2016-06-02Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Feldt, Robert
By organisation
Department of Software Engineering
In the same journal
Information and Software Technology
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

Altmetric score

Total: 121 hits
ReferencesLink to record
Permanent link

Direct link