Continuous Integration and Visual GUI Testing: Benefits and Drawbacks in Industrial Practice
2018 (English)In: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 172-181Conference paper, Published paper (Refereed)
Abstract [en]
Continuous integration (CI) is growing in industrial popularity, spurred on by market trends towards faster delivery and higher quality software. A key facilitator of CI is automated testing that should be executed, automatically, on several levels of system abstraction. However, many systems lack the interfaces required for automated testing. Others lack test automation coverage of the system under test's (SUT) graphical user interface (GUI) as it is shown to the user. One technique that shows promise to solve these challenges is Visual GUI Testing (VGT), which uses image recognition to stimulate and assert the SUT's behavior. Research has presented the technique's applicability and feasibility in industry but only limited support, from an academic setting, that the technique is applicable in a CI environment. In this paper we presents a study from an industrial design research study with the objective to help bridge the gap in knowledge regarding VGT's applicability in a CI environment in industry. Results, acquired from interviews, observations and quantitative analysis of 17.567 test executions, collected over 16 weeks, show that VGT provides similar benefits to other automated test techniques for CI. However, several significant drawbacks, such as high costs, are also identified. The study concludes that, although VGT is applicable in an industrial CI environment, its severe challenges require more research and development before the technique becomes efficient in practice. © 2018 IEEE.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2018. p. 172-181
Keywords [en]
Continuous Integration, Design Research, Empirical, Industrial Study, Visual GUI Testing, Automation, Behavioral research, Graphical user interfaces, Image recognition, Industrial research, Integration, Product design, Research and development management, Verification, Continuous integrations, Integration testing
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-16643DOI: 10.1109/ICST.2018.00026ISI: 000435006300016Scopus ID: 2-s2.0-85048401529ISBN: 9781538650127 (print)OAI: oai:DiVA.org:bth-16643DiVA, id: diva2:1228514
Conference
11th IEEE International Conference on Software Testing, Verification and Validation, ICST, Västerås
2018-06-282018-06-282021-03-31Bibliographically approved