Change search
Link to record
Permanent link

Direct link
BETA
Alégroth, Emil
Publications (10 of 12) Show all publications
Yu, L., Alégroth, E., Chatzipetrou, P. & Gorschek, T. (2020). Utilising CI environment for efficient and effective testing of NFRs. Information and Software Technology, 117, Article ID 106199.
Open this publication in new window or tab >>Utilising CI environment for efficient and effective testing of NFRs
2020 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 117, article id 106199Article in journal (Refereed) Published
Abstract [en]

Context: Continuous integration (CI) is a practice that aims to continuously verify quality aspects of a software intensive system both for functional and non-functional requirements (NFRs). Functional requirements are the inputs of development and can be tested in isolation, utilising either manual or automated tests. In contrast, some NFRs are difficult to test without functionality, for NFRs are often aspects of functionality and express quality aspects. Lacking this testability attribute makes NFR testing complicated and, therefore, underrepresented in industrial practice. However, the emergence of CI has radically affected software development and created new avenues for software quality evaluation and quality information acquisition. Research has, consequently, been devoted to the utilisation of this additional information for more efficient and effective NFR verification. Objective: We aim to identify the state-of-the-art of utilising the CI environment for NFR testing, hereinafter referred to as CI-NFR testing. Method: Through rigorous selection, from an initial set of 747 papers, we identified 47 papers that describe how NFRs are tested in a CI environment. Evidence-based analysis, through coding, is performed on the identified papers in this SLR. Results: Firstly, ten CI approaches are described by the papers selected, each describing different tools and nine different NFRs where reported to be tested. Secondly, although possible, CI-NFR testing is associated with eight challenges that adversely affect its adoption. Thirdly, the identified CI-NFR testing processes are tool-driven, but there is a lack of NFR testing tools that can be used in the CI environment. Finally, we proposed a CI framework for NFRs testing. Conclusion: A synthesised CI framework is proposed for testing various NFRs, and associated CI tools are also mapped. This contribution is valuable as results of the study also show that CI-NFR testing can help improve the quality of NFR testing in practices. © 2019

Place, publisher, year, edition, pages
Elsevier B.V., 2020
Keywords
Agile, CI, Continuous integration, DevOps, NFR, Non-functional requirement, SAFe, Scaled agile framework, Computer software selection and evaluation, Integral equations, Paper, Software design, Continuous integrations, Non-functional requirements, Quality control
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-18829 (URN)10.1016/j.infsof.2019.106199 (DOI)000496874400005 ()2-s2.0-85073572821 (Scopus ID)
Available from: 2019-10-31 Created: 2019-10-31 Last updated: 2019-12-11Bibliographically approved
Nass, M., Alégroth, E. & Feldt, R. (2019). Augmented testing: Industry feedback to shape a new testing technology. In: Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019: . Paper presented at 12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Xi'an, China, 22 April 2019 through 27 April 2019 (pp. 176-183). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Augmented testing: Industry feedback to shape a new testing technology
2019 (English)In: Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019, Institute of Electrical and Electronics Engineers Inc. , 2019, p. 176-183Conference paper, Published paper (Refereed)
Abstract [en]

Manual testing is the most commonly used approach in the industry today for acceptance-and system-testing of software applications. Test automation has been suggested to address drawbacks with manual testing but both test automation and manual testing have several challenges that limit their return of investment for system-and acceptance-test automation. Hence, there is still an industrial need for another approach to testing that can mitigate the challenges associated with system-and acceptance-testing and make it more efficient and cost effective for the industry. In this paper we present a novel technique we refer to as Augmented Testing (AT). AT is defined as testing through a visual layer between the tester and the System Under Test (SUT) that superimposes information on top of the GUI. We created a prototype for AT and performed an industrial workshop study with 10 software developers to get their perceived benefits and drawbacks of AT. The benefits and drawbacks will be useful for further development of the technique and prototype for AT. The workshop study identified more benefits than drawbacks with AT. Two of the identified benefits were: 'Know what to test and what has been tested' and 'Less manual work'. Due to these results, we believe that AT is a promising technique that deserves more research since it may provide industry with new benefits that current techniques lack. © 2019 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2019
Series
IEEE International Conference on Software Testing Verification and Validation Workshops
Keywords
Augmented Testing, Industrial Workshop Study, System Testing, Test Automation, Application programs, Automation, Cost effectiveness, Software prototyping, Software testing, System theory, Verification, Acceptance testing, Return of investments, Software applications, Software developer, Testing technology, Acceptance tests
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-18591 (URN)10.1109/ICSTW.2019.00048 (DOI)000477742600024 ()2-s2.0-85068385060 (Scopus ID)9781728108889 (ISBN)
Conference
12th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Xi'an, China, 22 April 2019 through 27 April 2019
Available from: 2019-09-09 Created: 2019-09-09 Last updated: 2019-09-10Bibliographically approved
Alégroth, E., Gorschek, T., Petersen, K. & Mattsson, M. (2019). Characteristics that affect Preference of Decision Models for Asset Selection: An Industrial Questionnaire Survey. Software quality journal
Open this publication in new window or tab >>Characteristics that affect Preference of Decision Models for Asset Selection: An Industrial Questionnaire Survey
2019 (English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367Article in journal, Editorial material (Refereed) Accepted
Abstract [en]

Modern software development relies on a combination of development and re-use of technical asset, e.g. software components, libraries and APIs.In the past, re-use was mostly conducted with internal assets but today external; open source, customer off-the-shelf (COTS) and assets developed through outsourcing are also common.This access to more asset alternatives presents new challenges regarding what assets to optimally chose and how to make this decision.To support decision-makers, decision-theory has been used to develop decision models for asset selection.However, very little industrial data has been presented in literature about the usefulness, or even perceived usefulness, of these models.Additionally, only limited information has been presented about what model characteristics that determine practitioner preference towards one model over another.

Objective: The objective of this work is to evaluate what characteristics of decision models for asset selection that determine industrial practitioner preference of a model when given the choice of a decision-model of high precision or a model with high speed.

Method: An industrial questionnaire survey is performed where a total of 33 practitioners, of varying roles, from 18 companies are tasked to compare two decision models for asset selection.Textual analysis and formal and descriptive statistics are then applied on the survey responses to answer the study's research questions.

Results: The study shows that the practitioners had clear preference towards the decision model that emphasised speed over the one that emphasised decision precision.This conclusion was determined to be because one of the models was perceived faster, had lower complexity, had, was more flexible in use for different decisions, was more agile how it could be used in operation, its emphasis on people, its emphasis on ``good enough'' precision and ability to fail fast if a decision was a failure.Hence, seven characteristics that the practitioners considered important for their acceptance of the model.

Conclusion: Industrial practitioner preference, which relates to acceptance, of decision models for asset selection is dependent on multiple characteristics that must be considered when developing a model for different types of decisions such as operational day-to-day decisions as well as more critical tactical or strategic decisions.The main contribution of this work are seven identified characteristics that can serve as industrial requirements for future research on decision models for asset selection.

Keywords
Decision-models, Characteristics, Industrial study, Survey, Model comparison
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-18687 (URN)
Projects
ORION
Funder
Knowledge Foundation
Available from: 2019-09-25 Created: 2019-09-25 Last updated: 2019-11-18Bibliographically approved
Alégroth, E., Gorschek, T., Petersen, K. & Mattsson, M. (2019). Characteristics that affect Preference of Decision Models for Asset Selection: An Industrial Questionnaire Survey - Appendix A: Questionnaire Introduction. Decision-making in Practice / Appendix B: Survey results.
Open this publication in new window or tab >>Characteristics that affect Preference of Decision Models for Asset Selection: An Industrial Questionnaire Survey - Appendix A: Questionnaire Introduction. Decision-making in Practice / Appendix B: Survey results
2019 (English)Data set, Aggregated data
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-18692 (URN)
Available from: 2019-09-25 Created: 2019-09-25 Last updated: 2019-11-18
Borg, M., Chatzipetrou, P., Wnuk, K., Alégroth, E., Gorschek, T., Papatheocharous, E., . . . Axelsson, J. (2019). Selecting component sourcing options: A survey of software engineering's broader make-or-buy decisions. Information and Software Technology, 112, 18-34
Open this publication in new window or tab >>Selecting component sourcing options: A survey of software engineering's broader make-or-buy decisions
Show others...
2019 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 112, p. 18-34Article in journal (Refereed) Published
Abstract [en]

Context: Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them from external sources. In CBSE, several different sourcing options are available: (1) developing software in-house, (2) outsourcing development, (3) buying commercial-off-the-shelf software, and (4) integrating open source software components. Objective: Unfortunately, there is little available research on how organizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Results: Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. Conclusion: We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking. © 2019

Place, publisher, year, edition, pages
Elsevier B.V., 2019
Keywords
Component-based software engineering, Decision making, Software architecture, Sourcing, Survey, Decision support systems, Open systems, Software design, Surveying, Surveys, Commercial off-the-shelf softwares, Industry practices, Make-or-buy decisions, Online questionnaire, Software development organizations, Software Evolution, Open source software
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-17871 (URN)10.1016/j.infsof.2019.03.015 (DOI)000469899100002 ()2-s2.0-85064013176 (Scopus ID)
Available from: 2019-05-02 Created: 2019-05-02 Last updated: 2019-06-20Bibliographically approved
Chatzipetrou, P., Alégroth, E., Papatheocharous, E., Borg, M., Gorschek, T. & Wnuk, K. (2018). Component selection in Software Engineering: Which attributes are the most important in the decision process?. In: EUROMICRO Conference Proceedings: . Paper presented at 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Prague (pp. 198-205). IEEE conference proceedings
Open this publication in new window or tab >>Component selection in Software Engineering: Which attributes are the most important in the decision process?
Show others...
2018 (English)In: EUROMICRO Conference Proceedings, IEEE conference proceedings, 2018, p. 198-205Conference paper, Published paper (Refereed)
Abstract [en]

Abstract— Component-based software engineering is a common approach to develop and evolve contemporary software systems where different component sourcing options are available: 1)Software developed internally (in-house), 2)Software developed outsourced, 3)Commercial of the shelf software, and 4) Open Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The object of the present study is to investigate what matters the most to industry practitioners during component selection. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using Compositional Data Analysis. The descriptive results showed that Cost was clearly considered the most important attribute during the component selection. Other important attributes for the practitioners were: Support of the component, Longevity prediction, and Level of off-the-shelf fit to product. Next an exploratory analysis was conducted based on the practitioners’ inherent characteristics. Nonparametric tests and biplots were used. It seems that smaller organizations and more immature products focus on different attributes than bigger organizations and mature products which focus more on Cost

Place, publisher, year, edition, pages
IEEE conference proceedings, 2018
Series
EUROMICRO Conference Proceedings, ISSN 1089-6503
Keywords
Component - based software engineering; Decision making; Compositional Data Analysis; Cumulative voti
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-17057 (URN)10.1109/SEAA.2018.00039 (DOI)000450238900030 ()978-1-5386-7383-6 (ISBN)
Conference
44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Prague
Funder
Knowledge Foundation, 20140218
Available from: 2018-09-27 Created: 2018-09-27 Last updated: 2018-12-13Bibliographically approved
Alégroth, E., Karlsson, A. & Radway, A. (2018). Continuous Integration and Visual GUI Testing: Benefits and Drawbacks in Industrial Practice. In: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018: . Paper presented at 11th IEEE International Conference on Software Testing, Verification and Validation, ICST, Västerås (pp. 172-181). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Continuous Integration and Visual GUI Testing: Benefits and Drawbacks in Industrial Practice
2018 (English)In: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 172-181Conference paper, Published paper (Refereed)
Abstract [en]

Continuous integration (CI) is growing in industrial popularity, spurred on by market trends towards faster delivery and higher quality software. A key facilitator of CI is automated testing that should be executed, automatically, on several levels of system abstraction. However, many systems lack the interfaces required for automated testing. Others lack test automation coverage of the system under test's (SUT) graphical user interface (GUI) as it is shown to the user. One technique that shows promise to solve these challenges is Visual GUI Testing (VGT), which uses image recognition to stimulate and assert the SUT's behavior. Research has presented the technique's applicability and feasibility in industry but only limited support, from an academic setting, that the technique is applicable in a CI environment. In this paper we presents a study from an industrial design research study with the objective to help bridge the gap in knowledge regarding VGT's applicability in a CI environment in industry. Results, acquired from interviews, observations and quantitative analysis of 17.567 test executions, collected over 16 weeks, show that VGT provides similar benefits to other automated test techniques for CI. However, several significant drawbacks, such as high costs, are also identified. The study concludes that, although VGT is applicable in an industrial CI environment, its severe challenges require more research and development before the technique becomes efficient in practice. © 2018 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
Continuous Integration, Design Research, Empirical, Industrial Study, Visual GUI Testing, Automation, Behavioral research, Graphical user interfaces, Image recognition, Industrial research, Integration, Product design, Research and development management, Verification, Continuous integrations, Integration testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-16643 (URN)10.1109/ICST.2018.00026 (DOI)000435006300016 ()2-s2.0-85048401529 (Scopus ID)9781538650127 (ISBN)
Conference
11th IEEE International Conference on Software Testing, Verification and Validation, ICST, Västerås
Available from: 2018-06-28 Created: 2018-06-28 Last updated: 2018-07-05Bibliographically approved
Ardito, L., Coppola, R., Torchiano, M. & Alégroth, E. (2018). Towards automated translation between generations of GUI-based tests for mobile devices. In: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops: . Paper presented at 2018 International Symposium on Software Testing and Analysis, ISSTA 2018, Amsterdam, 16 July 2018 through 21 July 2018 (pp. 46-53). Association for Computing Machinery, Inc
Open this publication in new window or tab >>Towards automated translation between generations of GUI-based tests for mobile devices
2018 (English)In: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops, Association for Computing Machinery, Inc , 2018, p. 46-53Conference paper, Published paper (Refereed)
Abstract [en]

Market demands for faster delivery and higher software quality are progressively becoming more stringent. A key hindrance for software companies to meet such demands is how to test the software due to to the intrinsic costs of development, maintenance and evolution of testware. Especially since testware should be defined, and aligned, with all layers of system under test (SUT), including all graphical user interface (GUI) abstraction levels. These levels can be tested with different generations of GUI-based test approaches, where 2nd generation, or Layout-based, tests leverage GUI properties and 3rd generation, or Visual, tests make use of image recognition. The two approaches provide different benefits and drawbacks and are seldom used together because of the aforementioned costs, despite growing academic evidence of the complementary benefits. In this work we propose the proof of concept of a novel two-step translation approach for Android GUI testing that we aim to implement, where a translator first creates a technology independent script with actions and elements of the GUI, and then translates it to a script with the syntax chosen by the user. The approach enables users to translate Layout-based to Visual scripts and vice versa, to gain the benefits (e.g. robustness, speed and ability to emulate the user) of both generations, whilst minimizing the drawbacks (e.g. development and maintenance costs). We outline our approach from a technical perspective, discuss some of the key challenges with the realization of our approach, evaluate the feasibility and the advantages provided by our approach on an open-source Android application, and discuss the potential industrial impact of this work. © 2018 ACM.

Place, publisher, year, edition, pages
Association for Computing Machinery, Inc, 2018
Keywords
Android (operating system), Computer software selection and evaluation, Costs, Graphical user interfaces, Image recognition, Open source software, Testing, Abstraction level, Android applications, Automated translation, Graphical user interfaces (GUI), Proof of concept, Software Quality, System under test, Technology independent, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-17623 (URN)10.1145/3236454.3236488 (DOI)2-s2.0-85060987331 (Scopus ID)9781450359399 (ISBN)
Conference
2018 International Symposium on Software Testing and Analysis, ISSTA 2018, Amsterdam, 16 July 2018 through 21 July 2018
Available from: 2019-02-21 Created: 2019-02-21 Last updated: 2019-02-21Bibliographically approved
Alégroth, E. & Feldt, R. (2017). On the long-term use of visual gui testing in industrial practice: a case study. Journal of Empirical Software Engineering, 22(6), 2937-2971
Open this publication in new window or tab >>On the long-term use of visual gui testing in industrial practice: a case study
2017 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 22, no 6, p. 2937-2971Article in journal (Refereed) Published
Abstract [en]

Visual GUI Testing (VGT) is a tool-driven technique for automated GUI-based testing that uses image recognition to interact with and assert the correctness of the behavior of a system through its GUI as it is shown to the user. The technique’s applicability, e.g. defect-finding ability, and feasibility, e.g. time to positive return on investment, have been shown through empirical studies in industrial practice. However, there is a lack of studies that evaluate the usefulness and challenges associated with VGT when used long-term (years) in industrial practice. This paper evaluates how VGT was adopted, applied and why it was abandoned at the music streaming application development company, Spotify, after several years of use. A qualitative study with two workshops and five well chosen employees is performed at the company, supported by a survey, which is analyzed with a grounded theory approach to answer the study’s three research questions. The interviews provide insights into the challenges, problems and limitations, but also benefits, that Spotify experienced during the adoption and use of VGT. However, due to the technique’s drawbacks, VGT has been abandoned for a new technique/framework, simply called the Test interface. The Test interface is considered more robust and flexible for Spotify’s needs but has several drawbacks, including that it does not test the actual GUI as shown to the user like VGT does. From the study’s results it is concluded that VGT can be used long-term in industrial practice but it requires organizational change as well as engineering best practices to be beneficial. Through synthesis of the study’s results, and results from previous work, a set of guidelines are presented that aim to aid practitioners to adopt and use VGT in industrial practice. However, due to the abandonment of the technique, future research is required to analyze in what types of projects the technique is, and is not, long-term viable. To this end, we also present Spotify’s Test interface solution for automated GUI-based testing and conclude that it has its own benefits and drawbacks.

Place, publisher, year, edition, pages
Springer, 2017
Keywords
Automated testing, Case study, Guidelines, Industrial, Visual GUI testing, Automation, Image recognition, Industrial management, Engineering best practice, Grounded theory approach, Industrial practices, Organizational change, Graphical user interfaces
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13894 (URN)10.1007/s10664-016-9497-6 (DOI)000411865400006 ()2-s2.0-85010807231 (Scopus ID)
Available from: 2017-02-22 Created: 2017-02-22 Last updated: 2018-01-13Bibliographically approved
Alégroth, E., Matsuki, S., Vos, T. & Akemine, K. (2017). Overview of the ICST International Software Testing Contest. In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017: . Paper presented at 10th IEEE International Conference on Software Testing, Verification and Validation, ICST,Tokyo (pp. 550-551). IEEE Computer Society
Open this publication in new window or tab >>Overview of the ICST International Software Testing Contest
2017 (English)In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, IEEE Computer Society, 2017, p. 550-551Conference paper, Published paper (Refereed)
Abstract [en]

In the software testing contest, practitioners and researcher's are invited to test their test approaches against similar approaches to evaluate pros and cons and which is perceivably the best. The 2017 iteration of the contest focused on Graphical User Interface-driven testing, which was evaluated on the testing tool TESTONA. The winner of the competition was announced at the closing ceremony of the international conference on software testing (ICST), 2017. © 2017 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2017
Keywords
Automated testing, Comptetition, Gui testing, Graphical user interfaces, User interfaces, Verification, Testing tools, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-14898 (URN)10.1109/ICST.2017.67 (DOI)000403393600060 ()2-s2.0-85020754805 (Scopus ID)9781509060313 (ISBN)
Conference
10th IEEE International Conference on Software Testing, Verification and Validation, ICST,Tokyo
Available from: 2017-07-06 Created: 2017-07-06 Last updated: 2018-01-13Bibliographically approved
Organisations

Search in DiVA

Show all publications