Change search
Refine search result
1 - 41 of 41
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Extending the Boundaries of Higher Education through Digitalization: On the best practices of Onlineand Blended Learning2020Report (Other (popular science, discussion, etc.))
    Abstract [en]

    Accessibility of higher education has never been more important and using online teaching, or e-learning, is a suitable way of achieving this access. However, online education presents new challenges for teachers, which requires best practices to overcome.

    Download full text (pdf)
    Emil Alegroth_Extending the boundaries of Higher Education
    Download (pdf)
    Presentation_Extending the Boundaries of Higher Education through Digitalization
  • 2.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ardito, Luca
    Politecnico di Torino, Corso Duca degli Abruzzi, ITA.
    Coppola, Riccardo
    Politecnico di Torino, Corso Duca degli Abruzzi, ITA.
    Feldt, Robert
    Chalmers University of Technology, SWE.
    Special issue on new generations of UI testing2021In: Software testing, verification & reliability, ISSN 0960-0833, E-ISSN 1099-1689, Vol. 31, no 3, article id e1770Article in journal (Other academic)
    Download full text (pdf)
    fulltext
  • 3.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Feldt, Robert
    Chalmers, SWE.
    On the long-term use of visual gui testing in industrial practice: a case study2017In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 22, no 6, p. 2937-2971Article in journal (Refereed)
    Abstract [en]

    Visual GUI Testing (VGT) is a tool-driven technique for automated GUI-based testing that uses image recognition to interact with and assert the correctness of the behavior of a system through its GUI as it is shown to the user. The technique’s applicability, e.g. defect-finding ability, and feasibility, e.g. time to positive return on investment, have been shown through empirical studies in industrial practice. However, there is a lack of studies that evaluate the usefulness and challenges associated with VGT when used long-term (years) in industrial practice. This paper evaluates how VGT was adopted, applied and why it was abandoned at the music streaming application development company, Spotify, after several years of use. A qualitative study with two workshops and five well chosen employees is performed at the company, supported by a survey, which is analyzed with a grounded theory approach to answer the study’s three research questions. The interviews provide insights into the challenges, problems and limitations, but also benefits, that Spotify experienced during the adoption and use of VGT. However, due to the technique’s drawbacks, VGT has been abandoned for a new technique/framework, simply called the Test interface. The Test interface is considered more robust and flexible for Spotify’s needs but has several drawbacks, including that it does not test the actual GUI as shown to the user like VGT does. From the study’s results it is concluded that VGT can be used long-term in industrial practice but it requires organizational change as well as engineering best practices to be beneficial. Through synthesis of the study’s results, and results from previous work, a set of guidelines are presented that aim to aid practitioners to adopt and use VGT in industrial practice. However, due to the abandonment of the technique, future research is required to analyze in what types of projects the technique is, and is not, long-term viable. To this end, we also present Spotify’s Test interface solution for automated GUI-based testing and conclude that it has its own benefits and drawbacks.

  • 4.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gonzalez-Huerta, Javier
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Towards a mapping of software technical debt onto testware2017In: Proceedings - 43rd Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2017, Institute of Electrical and Electronics Engineers Inc. , 2017, p. 404-411, article id 8051379Conference paper (Refereed)
    Abstract [en]

    Technical Debt (TD) is a metaphor used to explain the negative impacts that sub-optimal design decisions have in the long-term perspective of a software project. Although TD is acknowledged by both researchers and practitioners to have strong negative impact on Software development, its study on Testware has so far been very limited. A gap in knowledge that is important to address due to the growing popularity of Testware (scripted automated testing) in software development practice.In this paper we present a mapping analysis that connects 21 well-known, Software, object-oriented TD items to Testware, establishing them as Testware Technical Debt (TTD) items. The analysis indicates that most Software TD items are applicable or observable as TTD items, often in similar form and with roughly the same impact as for Software artifacts (e.g. reducing quality of the produced artifacts, lowering the effectiveness and efficiency of the development process whilst increasing costs). In the analysis, we also identify three types of connections between software TD and TTD items with varying levels of impact and criticality. Additionally, the study finds support for previous research results in which specific TTD items unique to Testware were identified. Finally, the paper outlines several areas of future research into TTD. © 2017 IEEE.

  • 5.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Mattsson, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Characteristics that affect Preference of Decision Models for Asset Selection: An Industrial Questionnaire Survey2020In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 28, no 4, p. 1675-1707Article in journal (Refereed)
    Abstract [en]

    Modern software development relies on a combination of development and re-use of technical asset, e.g. software components, libraries and APIs.In the past, re-use was mostly conducted with internal assets but today external; open source, customer off-the-shelf (COTS) and assets developed through outsourcing are also common.This access to more asset alternatives presents new challenges regarding what assets to optimally chose and how to make this decision.To support decision-makers, decision-theory has been used to develop decision models for asset selection.However, very little industrial data has been presented in literature about the usefulness, or even perceived usefulness, of these models.Additionally, only limited information has been presented about what model characteristics that determine practitioner preference towards one model over another.

    Objective: The objective of this work is to evaluate what characteristics of decision models for asset selection that determine industrial practitioner preference of a model when given the choice of a decision-model of high precision or a model with high speed.

    Method: An industrial questionnaire survey is performed where a total of 33 practitioners, of varying roles, from 18 companies are tasked to compare two decision models for asset selection.Textual analysis and formal and descriptive statistics are then applied on the survey responses to answer the study's research questions.

    Results: The study shows that the practitioners had clear preference towards the decision model that emphasised speed over the one that emphasised decision precision.This conclusion was determined to be because one of the models was perceived faster, had lower complexity, had, was more flexible in use for different decisions, was more agile how it could be used in operation, its emphasis on people, its emphasis on ``good enough'' precision and ability to fail fast if a decision was a failure.Hence, seven characteristics that the practitioners considered important for their acceptance of the model.

    Conclusion: Industrial practitioner preference, which relates to acceptance, of decision models for asset selection is dependent on multiple characteristics that must be considered when developing a model for different types of decisions such as operational day-to-day decisions as well as more critical tactical or strategic decisions.The main contribution of this work are seven identified characteristics that can serve as industrial requirements for future research on decision models for asset selection.

    Download full text (pdf)
    fulltext
  • 6.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Mattsson, Michael
    Characteristics that affect Preference of Decision Models for Asset Selection: An Industrial Questionnaire Survey - Appendix A: Questionnaire Introduction. Decision-making in Practice / Appendix B: Survey results2019Data set
    Download full text (pdf)
    Appendix A: Questionnaire Introduction Decision-making in Practice
    Download full text (csv)
    Appendix B: Survey results
  • 7.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gustafsson, Johan
    SAAB AB, SWE.
    Ivarsson, Henrik
    SAAB AB, SWE.
    Feldt, Robert
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Replicating Rare Software Failures with Exploratory Visual GUI Testing2017In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 34, no 5, p. 53-59, article id 8048660Article in journal (Refereed)
    Abstract [en]

    Saab AB developed software that had a defect that manifested itself only after months of continuous system use. After years of customer failure reports, the defect still persisted, until Saab developed failure replication based on visual GUI testing. © 1984-2012 IEEE.

  • 8.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Karl, Kristian
    Spotify, SWE.
    Rosshagen, Helena
    AddQ, SWE.
    Helmfridsson, Tomas
    AddQ, SWE.
    Olsson, Nils
    ArcticBlue, SWE.
    Practitioners' best practices to Adopt, Use or Abandon Model-based Testing with Graphical models for Software-intensive Systems2022In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 27, no 5, article id 103Article in journal (Refereed)
    Abstract [en]

    Model-based testing (MBT) has been extensively researched for software-intensive systems but, despite the academic interest, adoption of the technique in industry has been sparse. This phenomenon has been observed by our industrial partners for MBT with graphical models. They perceive one cause to be a lack of evidence-based MBT guidelines that, in addition to technical guidelines, also take non-technical aspects into account. This hypothesis is supported by a lack of such guidelines in the literature. Objective: The objective of this study is to elicit, and synthesize, MBT experts' best practices for MBT with graphical models. The results aim to give guidance to practitioners and aspire to give researchers new insights to inspire future research. Method: An interview survey is conducted using deep, semi-structured, interviews with an international sample of 17 MBT experts, in different roles, from software industry. Interview results are synthesised through semantic equivalence analysis and verified by MBT experts from industrial practice. Results: 13 synthesised conclusions are drawn from which 23 best-practice guidelines are derived for the adoption, use and abandonment of the technique. In addition, observations and expert insights are discussed that help explain the lack of wide-spread adoption of MBT with graphical models in industrial practice. Conclusions: Several technical aspects of MBT are covered by the results as well as conclusions that cover process- and organizational factors. These factors relate to the mindset, knowledge, organization, mandate and resources that enable the technique to be used effectively within an organization. The guidelines presented in this work complement existing knowledge and, as a primary objective, provide guidance for industrial practitioners to better succeed with MBT with graphical models.

    Download full text (pdf)
    fulltext
  • 9.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Karlsson, Arvid
    Cilbuper IT, Gothenburg, SWE.
    Radway, Alexander
    Techship Krokslatts Fabriker, SWE.
    Continuous Integration and Visual GUI Testing: Benefits and Drawbacks in Industrial Practice2018In: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 172-181Conference paper (Refereed)
    Abstract [en]

    Continuous integration (CI) is growing in industrial popularity, spurred on by market trends towards faster delivery and higher quality software. A key facilitator of CI is automated testing that should be executed, automatically, on several levels of system abstraction. However, many systems lack the interfaces required for automated testing. Others lack test automation coverage of the system under test's (SUT) graphical user interface (GUI) as it is shown to the user. One technique that shows promise to solve these challenges is Visual GUI Testing (VGT), which uses image recognition to stimulate and assert the SUT's behavior. Research has presented the technique's applicability and feasibility in industry but only limited support, from an academic setting, that the technique is applicable in a CI environment. In this paper we presents a study from an industrial design research study with the objective to help bridge the gap in knowledge regarding VGT's applicability in a CI environment in industry. Results, acquired from interviews, observations and quantitative analysis of 17.567 test executions, collected over 16 weeks, show that VGT provides similar benefits to other automated test techniques for CI. However, several significant drawbacks, such as high costs, are also identified. The study concludes that, although VGT is applicable in an industrial CI environment, its severe challenges require more research and development before the technique becomes efficient in practice. © 2018 IEEE.

  • 10.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Matsuki, Shinsuke
    Veriserve Corporation, JPN.
    Vos, Tanja
    Open University of the Netherlands, NLD.
    Akemine, Kinji
    Nippon Telegraph and Telephone Corporation, JPN.
    Overview of the ICST International Software Testing Contest2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, IEEE Computer Society, 2017, p. 550-551Conference paper (Refereed)
    Abstract [en]

    In the software testing contest, practitioners and researcher's are invited to test their test approaches against similar approaches to evaluate pros and cons and which is perceivably the best. The 2017 iteration of the contest focused on Graphical User Interface-driven testing, which was evaluated on the testing tool TESTONA. The winner of the competition was announced at the closing ceremony of the international conference on software testing (ICST), 2017. © 2017 IEEE.

  • 11.
    Alégroth, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersén, Elin
    Linköping University, SWE.
    Tinnerholm, John
    Linköping University, SWE.
    A Failed attempt at creating Guidelines for Visual GUI Testing: An industrial case study2021In: Proceedings - 2021 IEEE 14th International Conference on Software Testing, Verification and Validation, ICST 2021, Institute of Electrical and Electronics Engineers Inc. , 2021, p. 340-350, article id 9438551Conference paper (Refereed)
    Abstract [en]

    Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE.

    Download full text (pdf)
    fulltext
  • 12.
    Ardito, Luca
    et al.
    Politecnico di Torino, ITA.
    Coppola, Riccardo
    Politecnico di Torino, ITA.
    Torchiano, Marco
    Politecnico di Torino, ITA.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Towards automated translation between generations of GUI-based tests for mobile devices2018In: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops, Association for Computing Machinery, Inc , 2018, p. 46-53Conference paper (Refereed)
    Abstract [en]

    Market demands for faster delivery and higher software quality are progressively becoming more stringent. A key hindrance for software companies to meet such demands is how to test the software due to to the intrinsic costs of development, maintenance and evolution of testware. Especially since testware should be defined, and aligned, with all layers of system under test (SUT), including all graphical user interface (GUI) abstraction levels. These levels can be tested with different generations of GUI-based test approaches, where 2nd generation, or Layout-based, tests leverage GUI properties and 3rd generation, or Visual, tests make use of image recognition. The two approaches provide different benefits and drawbacks and are seldom used together because of the aforementioned costs, despite growing academic evidence of the complementary benefits. In this work we propose the proof of concept of a novel two-step translation approach for Android GUI testing that we aim to implement, where a translator first creates a technology independent script with actions and elements of the GUI, and then translates it to a script with the syntax chosen by the user. The approach enables users to translate Layout-based to Visual scripts and vice versa, to gain the benefits (e.g. robustness, speed and ability to emulate the user) of both generations, whilst minimizing the drawbacks (e.g. development and maintenance costs). We outline our approach from a technical perspective, discuss some of the key challenges with the realization of our approach, evaluate the feasibility and the advantages provided by our approach on an open-source Android application, and discuss the potential industrial impact of this work. © 2018 ACM.

  • 13.
    Bauer, Andreas
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    We Tried and Failed: An Experience Report on a Collaborative Workflow for GUI-based Testing2023In: Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 1-9Conference paper (Refereed)
    Abstract [en]

    Modern software development is a team-based effort supported by tools, processes, and practices. One integral part is automated testing, where developers incorporate automated tests on multiple levels of system abstraction, from low-level unit tests to high-level system tests and Graphical User Interface (GUI) tests. Furthermore, the common practices of code reviews allow collaboration on artifacts based on discussions that improve the artifact's quality and to share information within the team. However, the characteristics of GUI-based tests, due to the level of abstraction and visual elements, introduce additional requirements and complexities compared to code or lower-level test code review, delimiting the practice benefits.The objective of this work is to propose a tool-supported workflow that enables active collaboration among stakeholders and improves the efficiency and effectiveness of team-based development of GUI-based tests.To evaluate the workflow, and show proof of concept, a technical demonstrator for merging of GUI-based tests was to be developed. However, during its development, we encountered several unforeseen challenges that forced us to halt its development. We report the negative results from this development and the main challenges we encountered, as well as the rationale and the decisions we took towards this workflow.In conclusion, this work presents a negative research result on a failed attempt to propose a tool-supported workflow that enables active collaboration on GUI-based tests. The outcome and learnings of this work are intended to guide future research and prevent researchers from falling into the same pitfalls we did. © 2023 IEEE.

    Download full text (pdf)
    fulltext
  • 14.
    Bauer, Andreas
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Coppola, Ricardo
    Politecnico di Torino, Italy.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Code review guidelines for GUI-based testing artifacts2023In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 163, article id 107299Article, review/survey (Refereed)
    Abstract [en]

    Context: Review of software artifacts, such as source or test code, is a common practice in industrial practice. However, although review guidelines are available for source and low-level test code, for GUI-based testing artifacts, such guidelines are missing. Objective: The goal of this work is to define a set of guidelines from literature about production and test code, that can be mapped to GUI-based testing artifacts. Method: A systematic literature review is conducted, using white and gray literature to identify guidelines for source and test code. These synthesized guidelines are then mapped, through examples, to create actionable, and applicable, guidelines for GUI-based testing artifacts. Results: The results of the study are 33 guidelines, summarized in nine guideline categories, that are successfully mapped as applicable to GUI-based testing artifacts. Of the collected literature, only 10 sources contained test-specific code review guidelines. These guideline categories are: perform automated checks, use checklists, provide context information, utilize metrics, ensure readability, visualize changes, reduce complexity, check conformity with the requirements and follow design principles and patterns. Conclusion: This pivotal set of guidelines provides an industrial contribution in filling the gap of general guidelines for review of GUI-based testing artifacts. Additionally, this work highlights, from an academic perspective, the need for future research in this area to also develop guidelines for other specific aspects of GUI-based testing practice, and to take into account other facets of the review process not covered by this work, such as reviewer selection. © 2023 The Author(s)

    Download full text (pdf)
    fulltext
  • 15.
    Bauer, Andreas
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Frattini, Julian
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Augmented Testing to support Manual GUI-based Regression Testing: An Empirical StudyManuscript (preprint) (Other academic)
    Abstract [en]

    Context: Manual graphical user interface (GUI) software testing presents a substantial part of the overall practiced testing efforts, despite various research efforts to further increase test automation. Augmented Testing (AT), a novel approach for GUI testing, aims to aid manual GUI-based testing through a tool-supported approach where an intermediary visual layer is rendered between the system under test (SUT) and the tester, superimposing relevant test information.

    Objective: The primary objective of this study is to gather empirical evidence regarding AT's efficiency compared to manual GUI-based regression testing. Existing studies involving testing approaches under the AT definition primarily focus on exploratory GUI testing, leaving a gap in the context of regression testing. As a secondary objective, we investigate AT's benefits, drawbacks, and usability issues when deployed with the demonstrator tool, Scout.

    Method: We conducted an experiment involving 13 industry professionals, from six companies, comparing AT to manual GUI-based regression testing. These results were complemented by interviews and Bayesian data analysis (BDA) of the study's quantitative results.

    Results: The results of the Bayesian data analysis revealed that the use of AT shortens test durations in 70% of the cases on average, concluding that AT is more efficient.When comparing the means of the total duration to perform all tests, AT reduced the test duration by 36% in total. Participant interviews highlighted nine benefits and eleven drawbacks of AT, while observations revealed four usability issues.

    Conclusion: This study makes an empirical contribution to understanding Augmented Testing, a promising approach to improve the efficiency of GUI-based regression testing in practice. Furthermore, it underscores the importance of continual refinements of AT.

  • 16.
    Borg, Markus
    et al.
    RISE SICS AB, SWE.
    Alegroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Runeson, Per
    Lunds Universitet, SWE.
    Software Engineers' Information Seeking Behavior in Change Impact Analysis: An Interview Study2017In: IEEE International Conference on Program Comprehension, IEEE Computer Society , 2017, p. 12-22Conference paper (Refereed)
    Abstract [en]

    Software engineers working in large projects must navigate complex information landscapes. Change Impact Analysis (CIA) is a task that relies on engineers' successful information seeking in databases storing, e.g., source code, requirements, design descriptions, and test case specifications. Several previous approaches to support information seeking are task-specific, thus understanding engineers' seeking behavior in specific tasks is fundamental. We present an industrial case study on how engineers seek information in CIA, with a particular focus on traceability and development artifacts that are not source code. We show that engineers have different information seeking behavior, and that some do not consider traceability particularly useful when conducting CIA. Furthermore, we observe a tendency for engineers to prefer less rigid types of support rather than formal approaches, i.e., engineers value support that allows flexibility in how to practically conduct CIA. Finally, due to diverse information seeking behavior, we argue that future CIA support should embrace individual preferences to identify change impact by empowering several seeking alternatives, including searching, browsing, and tracing. © 2017 IEEE.

  • 17.
    Borg, Markus
    et al.
    RISE, Sweden.
    Chatzipetrou, Panagiota
    Örebro University.
    Wnuk, Krzysztof
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Selecting Software Component Sourcing Options: Detailed Survey Description and Analysis2018Report (Other academic)
    Abstract [en]

    Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them fromexternal sources. In CBSE, several different sourcing options are available: 1) developing software in-house, 2) outsourcing development, 3) buying commercial-off-the-shelf software, and 4) integrating open source software components. Unfortunately, there is little available research on howorganizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.

  • 18.
    Borg, Markus
    et al.
    RISE Research Institutes of Sweden AB, SWE.
    Chatzipetrou, Panagiota
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Wnuk, Krzysztof
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Papatheocharous, Efi
    RISE Research Institutes of Sweden AB, SWE.
    Shah, Syed Muhammad Ali
    iZettle, SWE.
    Axelsson, Jakob
    RISE Research Institutes of Sweden AB, SWE.
    Selecting component sourcing options: A survey of software engineering's broader make-or-buy decisions2019In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 112, p. 18-34Article in journal (Refereed)
    Abstract [en]

    Context: Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them from external sources. In CBSE, several different sourcing options are available: (1) developing software in-house, (2) outsourcing development, (3) buying commercial-off-the-shelf software, and (4) integrating open source software components. Objective: Unfortunately, there is little available research on how organizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Results: Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. Conclusion: We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking. © 2019

    Download full text (pdf)
    fulltext
  • 19.
    Chatzipetrou, Panagiota
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Papatheocharous, Efi
    RISE SICS AB, SWE.
    Borg, Markus
    RISE SICS AB, SWE.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Wnuk, Krzysztof
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Component selection in Software Engineering: Which attributes are the most important in the decision process?2018In: EUROMICRO Conference Proceedings, IEEE conference proceedings, 2018, p. 198-205Conference paper (Refereed)
    Abstract [en]

    Abstract— Component-based software engineering is a common approach to develop and evolve contemporary software systems where different component sourcing options are available: 1)Software developed internally (in-house), 2)Software developed outsourced, 3)Commercial of the shelf software, and 4) Open Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The object of the present study is to investigate what matters the most to industry practitioners during component selection. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using Compositional Data Analysis. The descriptive results showed that Cost was clearly considered the most important attribute during the component selection. Other important attributes for the practitioners were: Support of the component, Longevity prediction, and Level of off-the-shelf fit to product. Next an exploratory analysis was conducted based on the practitioners’ inherent characteristics. Nonparametric tests and biplots were used. It seems that smaller organizations and more immature products focus on different attributes than bigger organizations and mature products which focus more on Cost

  • 20.
    Chatzipetrou, Panagiota
    et al.
    Orebro Univ, SWE.
    Papatheocharous, Efi
    RISE Res Inst Sweden AB, SWE.
    Wnuk, Krzysztof
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Blekinge Inst Technol, Software Engn Res Lab SERL, Karlskrona, Sweden..
    Borg, Markus
    RISE Res Inst Sweden AB, SWE.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Blekinge Inst Technol, Software Engn Res Lab SERL, Karlskrona, Sweden..
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Blekinge Inst Technol, Software Engn Res Lab SERL, Karlskrona, Sweden..
    Component attributes and their importance in decisions and component selection2020In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 28, no 2, p. 567-593Article in journal (Refereed)
    Abstract [en]

    Component-based software engineering is a common approach in the development and evolution of contemporary software systems. Different component sourcing options are available, such as: (1) Software developed internally (in-house), (2) Software developed outsourced, (3) Commercial off-the-shelf software, and (4) Open-Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The objective of this study is to investigate what matters the most to industry practitioners when they decide to select a component. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using compositional data analysis. The results of this exploratory analysis showed that cost was clearly considered to be the most important attribute for component selection. Other important attributes for the practitioners were: support of the component, longevity prediction, and level of off-the-shelf fit to product. Moreover, several practitioners still consider in-house software development to be the sole option when adding or replacing a component. On the other hand, there is a trend to complement it with other component sourcing options and, apart from cost, different attributes factor into their decision. Furthermore, in our analysis, nonparametric tests and biplots were used to further investigate the practitioners' inherent characteristics. It seems that smaller and larger organizations have different views on what attributes are the most important, and the most surprising finding is their contrasting views on the cost attribute: larger organizations with mature products are considerably more cost aware.

    Download full text (pdf)
    Component attributes and their importancein decisions and component selection
  • 21.
    Coppola, Riccardo
    et al.
    Politecnico di Torino, ITA.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A taxonomy of metrics for GUI-based testing research: A systematic literature review2022In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 152, article id 107062Article, review/survey (Refereed)
    Abstract [en]

    Context: GUI-based testing is a sub-field of software testing research that has emerged in the last three decades. GUI-based testing techniques focus on verifying the functional conformance of the system under test (SUT) through its graphical user interface. However, despite the research domains growth, studies in the field have low reproducibility and comparability. One observed cause of these phenomena is identified as a lack of research rigor and commonly used metrics, including coverage metrics. Objective: We aim to identify the most commonly used metrics in the field and formulate a taxonomy of coverage metrics for GUI-based testing research. Method: We adopt an evidence-based approach to build the taxonomy through a systematic literature review of studies in the GUI-based testing domain. Identified papers are then analyzed with Open and Axial Coding techniques to identify hierarchical and mutually exclusive categories of metrics with common characteristics, usages, and applications. Results: Through the analysis of 169 papers and 315 metric definitions, we obtained a taxonomy with 55 codes (common names for metrics), 17 metric categories, and 4 higher level categories: Functional Level, GUI Level, Model Level and Code Level. We measure a higher number of mentions of Model and Code level metrics over Functional and GUI level metrics. Conclusions: We propose a taxonomy for use in future GUI-based testing research to improve the general quality of studies in the domain. In addition, the taxonomy is perceived to help enable more replication studies as well as macro-analysis of the current body of research. © 2022 Elsevier B.V.

  • 22.
    Coppola, Riccardo
    et al.
    Polytechnic University of Turin, ITA.
    Ardito, Luca
    Polytechnic University of Turin, ITA.
    Torchiano, Marco
    Polytechnic University of Turin, ITA.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Translation from layout-based to visual android test scripts: An empirical evaluation2021In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 171, article id 110845Article in journal (Refereed)
    Abstract [en]

    Mobile GUI tests can be classified as layout-based – i.e. using GUI properties as locators – or Visual – i.e. using widgets’ screen captures as locators –. Visual test scripts require significant maintenance efforts to be kept aligned with the tested application as it evolves or it is ported to different devices. This work aims to conceptualize a translation-based approach to automatically derive Visual tests from existing layout-based counterparts or repair them when graphical changes occur, and to develop a tool that implements and validates the approach. We present TOGGLE, a tool that translates Espresso layout-based tests for Android apps to Visual tests that conform to either SikuliX, EyeAutomate, or a combination of the two tools’ syntax. An experiment is conducted to measure the precision of the translation approach, which is evaluated on maintenance tasks triggered by graphical changes due to device diversity. Our results demonstrate the feasibility of a translation-based approach, show that script portability to different devices is improved (from 32% to 93%), and indicate that translation can repair up to 90% of Visual locators in failing tests. GUI test translation mitigates challenges with Visual tests like maintenance effort and portability, enabling their wider use in industrial practice. © 2020 Elsevier Inc.

  • 23.
    Coppola, Riccardo
    et al.
    Politecnico di Torino, ITA.
    Ardito, Luca
    Politecnico di Torino, ITA.
    Torchiano, Marco
    Politecnico di Torino, ITA.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Translation from Visual to Layout-based Android Test Cases: A Proof of Concept2020In: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, p. 74-83, article id 9156007Conference paper (Refereed)
    Abstract [en]

    Context: 2nd generation (Layout-based) and 3rd generation (Visual) GUI testing are two approaches for testing mobile GUIs, both with individual benefits and drawbacks. Previous research has presented approaches to translate 2nd generation scripts to 3rd generation scripts but not the vice versa. Goal: The objective of this work is to provide Proof of Concept of the effectiveness of automatic translation between existing 3rd generation test scripts to 2nd generation test scripts. Method: A tool architecture is presented and implemented in a tool capable of translating most 3rd generation interactions with the GUI of an Android app into 2nd generation instructions and oracles for the Espresso testing tool.Results: We validate our approach on two test suites of our own creation, consisting of 30 test cases each. The measured success rate of the translation is 96.7% (58 working test cases out of 60 applications of the translator). Conclusion: The study provides support for the feasibility of a translation-based approach from 3rd to 2nd generation test cases. However, additional work is needed to make the approach applicable in real-world scenarios or larger open-source test suites. © 2020 IEEE.

  • 24.
    Coppola, Riccardo
    et al.
    Polytechnic University of Turin, Italy.
    Fulcini, Tommaso
    Polytechnic University of Turin, Italy.
    Ardito, Luca
    Polytechnic University of Turin, Italy.
    Torchiano, Marco
    Polytechnic University of Turin, Italy.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    On Effectiveness and Efficiency of Gamified Exploratory GUI Testing2023In: IEEE Transactions on Software Engineering, ISSN 0098-5589, E-ISSN 1939-3520Article in journal (Refereed)
    Abstract [en]

    Context: Gamification appears to improve enjoyment and quality of execution of software engineering activities, including software testing. Though commonly employed in industry, manual exploratory testing of web application GUIs was proven to be mundane and expensive. Gamification applied to that kind of testing activity has the potential to overcome its limitations, though no empirical research has explored this area yet.

    Goal: Collect preliminary insights on how gamification, when performed by novice testers, affects the effectiveness, efficiency, test case realism, and user experience in exploratory testing of web applications.

    Method: Common gamification features augment an existing exploratory testing tool: Final Score with Leaderboard, Injected Bugs, Progress Bar, and Exploration Highlights. The original tool and the gamified version are then compared in an experiment involving 144 participants. User experience is elicited using the Technology Acceptance Model (TAM) questionnaire instrument.

    Results: Statistical analysis identified several significant differences for metrics that represent the effectiveness and efficiency of tests showing an improvement in coverage when they were developed with gamification. Additionally, user experience is improved with gamification.

    Conclusions: Gamification of exploratory testing has a tangible effect on how testers create test cases for web applications. While the results are mixed, the effects are most beneficial and interesting and warrant more research in the future. Further research shall be aimed at confirming the presented results in the context of state-of-the-art testing tools and real-world development environments. 

  • 25.
    Fucci, Davide
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Axelsson, Thomas
    COMPANY, SWE.
    When traceability goes awry: An industrial experience report?2022In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 192, article id 111389Article in journal (Refereed)
    Abstract [en]

    The concept of traceability between artifacts is considered an enabler for software project success. This concept has received plenty of attention from the research community and is by many perceived to always be available in an industrial setting. In this industry-academia collaborative project, a team of researchers, supported by testing practitioners from a large telecommunication company, sought to investigate the partner company's issues related to software quality. However, it was soon identified that the fundamental traceability links between requirements and test cases were missing. This lack of traceability impeded the implementation of a solution to help the company deal with its quality issues. In this experience report, we discuss lessons learned about the practical value of creating and maintaining traceability links in complex industrial settings and provide a cautionary tale for researchers. (c) 2022 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

    Download full text (pdf)
    fulltext
  • 26.
    Kitamura, Takashi
    et al.
    National Institute of Advanced Industrial Science and Technology, JPN.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ramler, Rudolf
    Software Competence Center Hagenber, AUT.
    Industry-Academia Collaboration in Software Testing: An Overview of TAIC PART 20172017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017, Institute of Electrical and Electronics Engineers Inc. , 2017, p. 42-43Conference paper (Refereed)
    Abstract [en]

    Collaboration between industry and academia in software testing leads to improvement and innovation in industry, and it is the basis for achieving transferable and empirically evaluated results. Thus, the aim of TAIC PART is to forge collaboration between industry and academia on the challenging and exciting problem of real-world software testing. The workshop is promoted by representatives of both industry and academia, bringing together industrial software engineers and testers with researchers working on theory and practice of software testing. We present an overview of the 12th Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques (TAIC PART 2017) and its contributions. © 2017 IEEE.

  • 27.
    Lind, Emil
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gonzalez-Huerta, Javier
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Requirements Quality vs. Process and Stakeholders’ Well-Being: A Case of a Nordic Bank2023In: Software Quality: Higher Software Quality through Zero Waste Development / [ed] Mendez D., Winkler D., Winkler D., Kross J., Biffl S., Bergsmann J., Springer Science+Business Media B.V., 2023, Vol. 472, p. 17-37Conference paper (Refereed)
    Abstract [en]

    Requirements are key artefacts to describe the intended purpose of a software system. The quality of requirements is crucial for deciding what to do next, impacting the development process’ effectiveness and efficiency. However, we know very little about the connection between practitioners’ perceptions regarding requirements quality and its impact on the process or the feelings of the professionals involved in the development process. Objectives: This study investigates: i) How software development practitioners define requirements quality, ii) how the perceived quality of requirements impact process and stakeholders’ well-being, and iii) what are the causes and potential solutions for poor-quality requirements. Method: This study was performed as a descriptive interview study at a sub-organization of a Nordic bank that develops its own web and mobile apps. The data collection comprises interviews with 20 practitioners, including requirements engineers, developers, testers, and newly employed developers, with five interviewees from each group. Results: The results show that different roles have different views on what makes a requirement good quality. Participants highlighted that, in general, they experience negative emotions, more work, and overhead communication when they work with requirements they perceive to be of poor quality. The practitioners also describe positive effects on their performance and positive feelings when they work with requirements that they perceive to be good. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

  • 28.
    Nass, Michel
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Feldt, Robert
    Chalmers, SWE.
    Augmented testing: Industry feedback to shape a new testing technology2019In: Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019, Institute of Electrical and Electronics Engineers Inc. , 2019, p. 176-183Conference paper (Refereed)
    Abstract [en]

    Manual testing is the most commonly used approach in the industry today for acceptance-and system-testing of software applications. Test automation has been suggested to address drawbacks with manual testing but both test automation and manual testing have several challenges that limit their return of investment for system-and acceptance-test automation. Hence, there is still an industrial need for another approach to testing that can mitigate the challenges associated with system-and acceptance-testing and make it more efficient and cost effective for the industry. In this paper we present a novel technique we refer to as Augmented Testing (AT). AT is defined as testing through a visual layer between the tester and the System Under Test (SUT) that superimposes information on top of the GUI. We created a prototype for AT and performed an industrial workshop study with 10 software developers to get their perceived benefits and drawbacks of AT. The benefits and drawbacks will be useful for further development of the technique and prototype for AT. The workshop study identified more benefits than drawbacks with AT. Two of the identified benefits were: 'Know what to test and what has been tested' and 'Less manual work'. Due to these results, we believe that AT is a promising technique that deserves more research since it may provide industry with new benefits that current techniques lack. © 2019 IEEE.

  • 29.
    Nass, Michel
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. SERL, SWE.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. SERL, SWE.
    Feldt, Robert
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. SERL, SWE.
    On the Industrial Applicability of Augmented Testing: An Empirical Study2020In: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, p. 364-371, article id 9155725Conference paper (Refereed)
    Abstract [en]

    Testing applications with graphical user Interfaces (GUI) is an important but also a time-consuming task in practice. Tools and frameworks for GUI test automation can make the test execution more efficient and lower the manual labor required for regression testing. However, the test scripts used for automated GUI-based testing still require a substantial development effort and are often reported as sensitive to change, leading to frequent and costly maintenance. The efficiency of development, maintenance, and evolution of such tests are thereby dependent on the readability of scripts and the ease-of-use of test tools/frameworks in which the test scripts are defined. To address these shortcomings in existing state-of-practice techniques, a novel technique referred to as Augmented Testing (AT) has been proposed. AT is defined as testing the System Under Test (SUT) through an Augmented GUI that superimposes information on top of the SUT GUI. The Augmented GUI can provide the user with hints, test data, or other support while also observing and recording the tester's interactions. For this study, a prototype tool, called Scout, has been used that adheres to the AT concept that is evaluated in an industrial empirical study. In the evaluation, quasi-experiments and questionnaire surveys are performed in two workshops, with 12 practitioners from two Swedish companies (Ericsson and Inceptive). Results show that Scout can be used to create equivalent test cases faster, with statistical significance, than creating automated scripts in two popular state-of-practice tools. The study concludes that AT has cost-value benefits, applies to industrial-grade software, and overcomes several deficiencies of state-of-practice GUI testing technologies in terms of ease-of-use. © 2020 IEEE.

    Download full text (pdf)
    fulltext
  • 30.
    Nass, Michel
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Feldt, Robert
    Chalmers University of Technology, SWE.
    Why many challenges with GUI test automation (will) remain2021In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 138, article id 106625Article in journal (Refereed)
    Abstract [en]

    Context: Automated testing is ubiquitous in modern software development and used to verify requirement conformance on all levels of system abstraction, including the system's graphical user interface (GUI). GUI-based test automation, like other automation, aims to reduce the cost and time for testing compared to alternative, manual approaches. Automation has been successful in reducing costs for other forms of testing (like unit- or integration testing) in industrial practice. However, we have not yet seen the same convincing results for automated GUI-based testing, which has instead been associated with multiple technical challenges. Furthermore, the software industry has struggled with some of these challenges for more than a decade with what seems like only limited progress. Objective: This systematic literature review takes a longitudinal perspective on GUI test automation challenges by identifying them and then investigating why the field has been unable to mitigate them for so many years. Method: The review is based on a final set of 49 publications, all reporting empirical evidence from practice or industrial studies. Statements from the publications are synthesized, based on a thematic coding, into 24 challenges related to GUI test automation. Results: The most reported challenges were mapped chronologically and further analyzed to determine how they and their proposed solutions have evolved over time. This chronological mapping of reported challenges shows that four of them have existed for almost two decades. Conclusion: Based on the analysis, we discuss why the key challenges with GUI-based test automation are still present and why some will likely remain in the future. For others, we discuss possible ways of how the challenges can be addressed. Further research should focus on finding solutions to the identified technical challenges with GUI-based test automation that can be resolved or mitigated. However, in parallel, we also need to acknowledge and try to overcome non-technical challenges. © 2021

  • 31.
    Nass, Michel
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Feldt, Robert
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Coppola, Riccardo
    Politecnico di Torino, Italy.
    Robust web element identification for evolving applications by considering visual overlaps2023In: Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation, ICST 2023, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 258-268Conference paper (Refereed)
    Abstract [en]

    Fragile (i.e., non-robust) test execution is a common challenge for automated GUI-based testing of web applications as they evolve. Despite recent progress, there is still room for improvement since test execution failures caused by technical limitations result in unnecessary maintenance costs that limit its effectiveness and efficiency. One of the most reported technical challenges for web-based tests concerns how to reliably locate a web element used by a test script.This paper proposes the novel concept of Visually Overlapping Nodes (VON) that reduces fragility by utilizing the phenomenon that visual web elements (observed by the user) are constructed from multiple web-elements in the Document Object Model (DOM) that overlaps visually.We demonstrate the approach in a tool, VON Similo, which extends the state-of-the-art multi-locator approach (Similo) that is also used as the baseline for an experiment. In the experiment, a ground truth set of 1163 manually collected web element pairs, from different releases of the 40 most popular web applications on the internet, are used to compare the approaches' precision, recall, and accuracy.Our results show that VON Similo provides 94.7% accuracy in identifying a web element in a new release of the same SUT. In comparison, Similo provides 83.8% accuracy.These results demonstrate the applicability of the visually overlapping nodes concept/tool for web element localization in evolving web applications and contribute a novel way of thinking about web element localization in future research on GUI-based testing. © 2023 IEEE.

  • 32.
    Nass, Michel
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Feldt, Robert
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Leotta, Maurizio
    University Genoa, Italy.
    Ricca, Filippo
    University Genoa, Italy.
    Similarity-based Web Element Localization for Robust Test Automation2023In: ACM Transactions on Software Engineering and Methodology, ISSN 1049-331X, E-ISSN 1557-7392, Vol. 32, no 3, article id 75Article in journal (Refereed)
    Abstract [en]

    Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This article proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be 4 milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.

    Download full text (pdf)
    fulltext
  • 33.
    Nygren, Åse
    et al.
    Blekinge Institute of Technology, Education Development Unit.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Eriksson, Anna
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Pettersson, Eva
    Blekinge Institute of Technology, Education Development Unit.
    Does Previous Experience with Online Platforms Matter? A Survey about Online Learning across Study Programs2023In: Education Sciences, E-ISSN 2227-7102, Vol. 13, no 2, article id 181Article in journal (Refereed)
    Abstract [en]

    The COVID-19 pandemic has had a dramatic effect on society, including teaching within higher education that was forced to adapt to online teaching. Research on this phenomenon has looked at pedagogical methods as well as student perceptions of this way of teaching. However, to the best of our knowledge, no studies have looked at the wider perspective, within the entire student populous of a university, what students’ perceptions are and how these correlate with the students’ previous experiences and habits with online platforms, e.g., online streaming or social media. In this study, we perform a questionnaire survey with 431 responses with students from 20 programs at Blekinge Institute of technology. The survey responses are analyzed using descriptive statistics and qualitative analysis to draw its conclusions. Results show that there is no correlation between previous habits and student experience with online platforms in relation to online learning. Instead, other factors, e.g., teacher engagement, is found central for student learning and therefore important to consider for future research and development of online teaching methodologies. © 2023 by the authors.

    Download full text (pdf)
    fulltext
  • 34.
    Usman, Muhammad
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Felderer, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. University of Innsbruck, AUT.
    Unterkalmsteiner, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Klotins, Eriks
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Mendez, Daniel
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. fortiss GmbH, DEU.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Compliance Requirements in Large-Scale Software Development: An Industrial Case Study2020In: Lecture Notes in Computer Science / [ed] Morisio M.,Torchiano M.,Jedlitschka A., Springer-Verlag Tokyo Inc., 2020, Vol. 12562, p. 385-401Conference paper (Refereed)
    Abstract [en]

    Regulatory compliance is a well-studied area, including research on how to model, check, analyse, enact, and verify compliance of software. However, while the theoretical body of knowledge is vast, empirical evidence on challenges with regulatory compliance, as faced by industrial practitioners particularly in the Software Engineering domain, is still lacking. In this paper, we report on an industrial case study which aims at providing insights into common practices and challenges with checking and analysing regulatory compliance, and we discuss our insights in direct relation to the state of reported evidence. Our study is performed at Ericsson AB, a large telecommunications company, which must comply to both locally and internationally governing regulatory entities and standards such as GDPR. The main contributions of this work are empirical evidence on challenges experienced by Ericsson that complement the existing body of knowledge on regulatory compliance. © 2020, Springer Nature Switzerland AG.

    Download full text (pdf)
    fulltext
  • 35.
    Wohlin, Claes
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Papatheocharous, Efi
    RISE Research Institutes of Sweden AB, SWE.
    Carlson, Jan
    Mälardalen University, SWE.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Axelsson, Jakob
    Mälardalen University, SWE.
    Badampudi, Deepika
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Borg, Markus
    RISE Research Institutes of Sweden AB, SWE.
    Cicchetti, Antonio
    Mälardalen University, SWE.
    Ciccozzi, Federico
    Mälardalen University, SWE.
    Olsson, Thomas
    RISE Research Institutes of Sweden AB.
    Sentilles, Séverine
    Mälardalen University, SWE.
    Svahnberg, Mikael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Wnuk, Krzysztof
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Towards evidence-based decision-making for identification and usage of assets in composite software: A research roadmap2021In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 33, no 6, article id e2345Article in journal (Refereed)
    Abstract [en]

    Software engineering is decision intensive. Evidence-based software engineering is suggested for decision-making concerning the use of methods and technologies when developing software. Software development often includes the reuse of software assets, for example, open-source components. Which components to use have implications on the quality of the software (e.g., maintainability). Thus, research is needed to support decision-making for composite software. This paper presents a roadmap for research required to support evidence-based decision-making for choosing and integrating assets in composite software systems. The roadmap is developed as an output from a 5-year project in the area, including researchers from three different organizations. The roadmap is developed in an iterative process and is based on (1) systematic literature reviews of the area; (2) investigations of the state of practice, including a case survey and a survey; and (3) development and evaluation of solutions for asset identification and selection. The research activities resulted in identifying 11 areas in need of research. The areas are grouped into two categories: areas enabling evidence-based decision-making and those related to supporting the decision-making. The roadmap outlines research needs in these 11 areas. The research challenges and research directions presented in this roadmap are key areas for further research to support evidence-based decision-making for composite software. © 2021 The Authors. Journal of Software: Evolution and Process published by John Wiley & Sons Ltd.

    Download full text (pdf)
    fulltext
  • 36.
    Yu, Liang
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Ericsson AB.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Chatzipetrou, Panagiota
    Örebro University.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Automated NFR testing in Continuous Integration Environments: a multi-case study of Nordic companiesManuscript (preprint) (Other academic)
    Abstract [en]

    Context: Non-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software.Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation.Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of  Continuous Integration (CI).However, studies on how companies manage automated NFR testing through CI are limited.

    Objective: This study examines how automated NFR testing can be enabledand supported using CI environments in software development companies.

    Method: We performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners.

    Results: Maintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments.Testing practices, quality metrics, and challenges for measuring NFRs were reported.

    Conclusions: This study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice.

  • 37.
    Yu, Liang
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Chatzipetrou, Panagiota
    Örebro University School of Business.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Automated NFR testing in continuous integration environments: a multi-case study of Nordic companies2023In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 28, no 6, article id 144Article in journal (Refereed)
    Abstract [en]

    Context: Non-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software. Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation. Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of Continuous Integration (CI). However, studies on how companies manage automated NFR testing through CI are limited. Objective: This study examines how automated NFR testing can be enabled and supported using CI environments in software development companies. Method: We performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners. Results: Maintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments. Testing practices, quality metrics, and challenges for measuring NFRs were reported. Conclusions: This study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice. © 2023, The Author(s).

    Download full text (pdf)
    fulltext
  • 38.
    Yu, Liang
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Qvantel Sweden AB.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Chatzipetrou, Panagiota
    Örebro universitet, SWE.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Utilising CI environment for efficient and effective testing of NFRs2020In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 117, article id 106199Article in journal (Refereed)
    Abstract [en]

    Context: Continuous integration (CI) is a practice that aims to continuously verify quality aspects of a software intensive system both for functional and non-functional requirements (NFRs). Functional requirements are the inputs of development and can be tested in isolation, utilising either manual or automated tests. In contrast, some NFRs are difficult to test without functionality, for NFRs are often aspects of functionality and express quality aspects. Lacking this testability attribute makes NFR testing complicated and, therefore, underrepresented in industrial practice. However, the emergence of CI has radically affected software development and created new avenues for software quality evaluation and quality information acquisition. Research has, consequently, been devoted to the utilisation of this additional information for more efficient and effective NFR verification. Objective: We aim to identify the state-of-the-art of utilising the CI environment for NFR testing, hereinafter referred to as CI-NFR testing. Method: Through rigorous selection, from an initial set of 747 papers, we identified 47 papers that describe how NFRs are tested in a CI environment. Evidence-based analysis, through coding, is performed on the identified papers in this SLR. Results: Firstly, ten CI approaches are described by the papers selected, each describing different tools and nine different NFRs where reported to be tested. Secondly, although possible, CI-NFR testing is associated with eight challenges that adversely affect its adoption. Thirdly, the identified CI-NFR testing processes are tool-driven, but there is a lack of NFR testing tools that can be used in the CI environment. Finally, we proposed a CI framework for NFRs testing. Conclusion: A synthesised CI framework is proposed for testing various NFRs, and associated CI tools are also mapped. This contribution is valuable as results of the study also show that CI-NFR testing can help improve the quality of NFR testing in practices. © 2019

  • 39.
    Yu, Liang
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Ericsson AB.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Chatzipetrou, Panagiota
    Örebro University.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Utilizing Continuous Integration environments for evaluation of software quality attributes: a practical road mapManuscript (preprint) (Other academic)
    Abstract [en]

    The evaluation of quality attributes is often conducted in an ad-hoc manner in many companies.This practice, although often necessary to reduce cost, can lead to inefficient work and undetected quality defects.To mitigate these issues, approaches that make use of metrics/data from Continuous Integration (CI) environments have been proposed, offering many advantages, including timely feedback on code quality, early detection of quality issues, and visual representation of system quality trends.

    Despite these known benefits, there is a lack of frameworks that provide guidelines how to utilize CI environments, or their components, for quality attribute evaluation.Such guidelines are important since market demands on software quality factors, e.g., security or performance, are growing.Therefore, this study aims to explore the state-of-the-practice in using CI environments and identify shared guidelines underlying industrial practices.To achieve our research objectives, we conducted a multi-case study involving four selected companies and 22 interviews with industrial participants.

    In this paper, we first present a general model to offer an overview of how CI environments contribute to quality attribute evaluation.We then introduce common quality metrics used in the studied companies and from which CI components these metrics can be acquired.Finally, based on the study's findings, we propose a hierarchical and multi-layered decision support model for improving a CI environment's quality evaluation capabilities over time.

  • 40.
    Zabardast, Ehsan
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gonzalez-Huerta, Javier
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Šmite, Darja
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Fagerholm, Fabian
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A taxonomy of assets for the development of software-intensive products and services2023In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 202, article id 111701Article in journal (Refereed)
    Abstract [en]

    Context:Developing software-intensive products or services usually involves a plethora of software artefacts. Assets are artefacts intended to be used more than once and have value for organisations; examples include test cases, code, requirements, and documentation. During the development process, assets might degrade, affecting the effectiveness and efficiency of the development process. Therefore, assets are an investment that requires continuous management.

    Identifying assets is the first step for their effective management. However, there is a lack of awareness of what assets and types of assets are common in software-developing organisations. Most types of assets are understudied, and their state of quality and how they degrade over time have not been well-understood.

    Methods:We performed an analysis of secondary literature and a field study at five companies to investigate and identify assets to fill the gap in research. The results were analysed qualitatively and summarised in a taxonomy.

    Results:We present the first comprehensive, structured, yet extendable taxonomy of assets, containing 57 types of assets.

    Conclusions:The taxonomy serves as a foundation for identifying assets that are relevant for an organisation and enables the study of asset management and asset degradation concepts.

    Download full text (pdf)
    fulltext
  • 41.
    Zabardast, Ehsan
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gonzalez-Huerta, Javier
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Gorschek, Tony
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Šmite, Darja
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Alégroth, Emil
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Fagerholm, Fabian
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Asset Management Taxonomy: A Roadmap2021Manuscript (preprint) (Other academic)
    Abstract [en]

    Developing a software-intensive product or service can be a significant undertaking, associated with unique challenges in each project stage, from inception to development, delivery, maintenance, and evolution. Each step results in artefacts that are crucial for the project outcome, such as source-code and supporting deliverables, e.g., documentation.

    Artefacts which have inherent value for the organisation are assets, and as assets, they are subject to degradation. This degradation occurs over time, as artefacts age, and can be more immediate or slowly over a period of time, similar to the concept of technical debt. One challenge with the concept of assets is that it seems not to be well-understood and generally delimited to a few types of assets (often code-based), overlooking other equally important assets. 

    To bridge this gap, we have performed a study to formulate a structured taxonomy of assets. We use empirical data collected through industrial workshops and a literature review to ground the taxonomy. The taxonomy serves as foundations for concepts like asset degradation and asset management. The taxonomy can help contextualise, homogenise, extend the concept of technical debt, and serves as a conceptual framework for better identification, discussion, and utilisation of assets.

    Download full text (pdf)
    fulltext
1 - 41 of 41
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf