Change search
Link to record
Permanent link

Direct link
Alternative names
Publications (10 of 117) Show all publications
Minhas, N. M., Börstler, J. & Petersen, K. (2023). Checklists to support decision-making in regression testing. Journal of Systems and Software, 202, Article ID 111697.
Open this publication in new window or tab >>Checklists to support decision-making in regression testing
2023 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 202, article id 111697Article in journal (Refereed) Published
Abstract [en]

Context: Practitioners working in large-scale software development face many challenges in regression testing activities. One of the reasons is the lack of a structured regression testing process. In this regard, checklists can help practitioners keep track of essential regression testing activities and add structure to the regression testing process to a certain extent. Objective: This study aims to introduce regression testing checklists so test managers/teams can use them: (1) to assess whether test teams/members are ready to begin regression testing, and (2) to keep track of essential regression testing activities while planning and executing regression tests. Method: We used interviews, workshops, and questionnaires to design, evolve, and evaluate regression testing checklists. In total, 25 practitioners from 12 companies participated in creating the checklist. Twenty-three of them participated in checklists evolution and evaluation. Results: We identified activities practitioners consider significant while planning, performing, and analyzing regression testing. We designed regression testing checklists based on these activities to help practitioners make informed decisions during regression testing. With the help of practitioners, we evolved these checklists into two iterations. Finally, the practitioners provided feedback on the proposed checklists. All respondents think the proposed checklists are useful and customizable for their environments, and 80% think checklists cover aspects essential for regression testing. Conclusion: The proposed regression testing checklists can be useful for test managers to assess their team/team members’ readiness and decide when to start and stop regression testing. The checklists can be used to record the steps required while planning and executing regression testing. Further, these checklists can provide a basis for structuring the regression testing process in varying contexts. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Regression testing, Checklists, Test manager, Team readiness, Process improvement
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-23675 (URN)10.1016/j.jss.2023.111697 (DOI)000989289800001 ()2-s2.0-85153245617 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2022-09-18 Created: 2022-09-18 Last updated: 2023-06-12Bibliographically approved
Molléri, J. S., Mendes, E., Petersen, K. & Felderer, M. (2023). Determining a core view of research quality in empirical software engineering. Computer Standards & Interfaces, 84, Article ID 103688.
Open this publication in new window or tab >>Determining a core view of research quality in empirical software engineering
2023 (English)In: Computer Standards & Interfaces, ISSN 0920-5489, E-ISSN 1872-7018, Vol. 84, article id 103688Article in journal (Refereed) Published
Abstract [en]

Context: Research quality is intended to appraise the design and reporting of studies. It comprises a set of standards such as methodological rigor, practical relevance, and conformance to ethical standards. Depending on the perspective, different views of importance are given to the standards for research quality. Objective: To investigate the suitability of a conceptual model of research quality to Software Engineering (SE), from the perspective of researchers engaged in Empirical Software Engineering (ESE) research, in order to understand the core value of research quality. Method: We conducted a mixed-methods approach with two distinct group perspectives: (i) a research group; and (ii) the empirical SE research community. Our data collection approach comprised a questionnaire survey and a complementary focus group. We carried out a hierarchical voting prioritization to collect relative values for importance of standards for research quality. Results: In the context of this research, ‘internally valid’, ‘relevant research idea’, and ‘applicable results’ are perceived as the core standards for research quality in empirical SE. The alignment at the research group level was higher compared to that at the community level. Conclusion: The conceptual model was seen to express fairly the standards for research quality in the SE context. It presented limitations regarding its structure and components’ description, which resulted in an updated model. © 2022

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Alignment, Conceptual model, Research quality, Standards, Surveys, Core values, Data collection, Empirical Software Engineering, Ethical standards, Mixed method, Research communities, Research groups, Software engineering research, Software engineering
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-23706 (URN)10.1016/j.csi.2022.103688 (DOI)000870181900002 ()2-s2.0-85137713683 (Scopus ID)
Available from: 2022-10-03 Created: 2022-10-03 Last updated: 2023-12-04Bibliographically approved
Börstler, J., Ali, N. b. & Petersen, K. (2023). Double-counting in software engineering tertiary studies — An overlooked threat to validity. Information and Software Technology, 158, Article ID 107174.
Open this publication in new window or tab >>Double-counting in software engineering tertiary studies — An overlooked threat to validity
2023 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 158, article id 107174Article, review/survey (Refereed) Published
Abstract [en]

Context: Double-counting in a literature review occurs when the same data, population, or evidence is erroneously counted multiple times during synthesis. Detecting and mitigating the threat of double-counting is particularly challenging in tertiary studies. Although this topic has received much attention in the health sciences, it seems to have been overlooked in software engineering. Objective: We describe issues with double-counting in tertiary studies, investigate the prevalence of the issue in software engineering, and propose ways to identify and address the issue. Method: We analyze 47 tertiary studies in software engineering to investigate in which ways they address double-counting and whether double-counting might be a threat to validity in them. Results: In 19 of the 47 tertiary studies, double-counting might bias their results. Of those 19 tertiary studies, only 5 consider double-counting a threat to their validity, and 7 suggest strategies to address the issue. Overall, only 9 of the 47 tertiary studies, acknowledge double-counting as a potential general threat to validity for tertiary studies. Conclusions: Double-counting is an overlooked issue in tertiary studies in software engineering, and existing design and evaluation guidelines do not address it sufficiently. Therefore, we propose recommendations that may help to identify and mitigate double-counting in tertiary studies. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Population statistics, Bias, Double counting, Empirical, Guideline, Meta-review, Overview of review, Recommendation, Research method, Review of review, Tertiary review, Tertiary study, Umbrella review, Software engineering, Double-counting, Guidelines, Overview of reviews, Recommendations, Research methods, Review of reviews
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-24419 (URN)10.1016/j.infsof.2023.107174 (DOI)001005614800001 ()2-s2.0-85150795598 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2023-04-07 Created: 2023-04-07 Last updated: 2023-06-30Bibliographically approved
Börstler, J., Ali, N. b., Svensson, M. & Petersen, K. (2023). Investigating Acceptance Behavior in Software Engineering – Theoretical Perspectives. Journal of Systems and Software, 198, Article ID 111592.
Open this publication in new window or tab >>Investigating Acceptance Behavior in Software Engineering – Theoretical Perspectives
2023 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 198, article id 111592Article in journal (Refereed) Published
Abstract [en]

Background: Software engineering research aims to establish software development practice on a scientific basis. However, the evidence of the efficacy of technology is insufficient to ensure its uptake in industry. In the absence of a theoretical frame of reference, we mainly rely on best practices and expert judgment from industry-academia collaboration and software process improvement research to improve the acceptance of the proposed technology. Objective: To identify acceptance models and theories and discuss their applicability in the research of acceptance behavior related to software development.Method: We analyzed literature reviews within an interdisciplinary team to identify models and theories relevant to software engineering research. We further discuss acceptance behavior from the human information processing perspective of automatic and affect-driven processes (“fast” system 1 thinking) and rational and rule-governed processes (“slow” system 2 thinking). Results: We identified 30 potentially relevant models and theories. Several of them have been used in researching acceptance behavior in contexts related to software development, but few have been validated in such contexts. They use constructs that capture aspects of (automatic) system 1 and (rational) system 2 oriented processes. However, their operationalizations focus on system 2-oriented processes indicating a rational view of behavior, thus overlooking important psychological processes underpinning behavior. Conclusions: Software engineering research may use acceptance behavior models and theories more extensively to understand and predict practice adoption in the industry. Such theoretical foundations will help improve the impact of software engineering research. However, more consideration should be given to their validation, overlap, construct operationalization, and employed data collection mechanisms when using these models and theories.

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Acceptance behavior, dual process theory, technology acceptance, theory, TAM, UTAUT, TPB
National Category
Software Engineering Psychology
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-24132 (URN)10.1016/j.jss.2022.111592 (DOI)000915632900001 ()2-s2.0-85146227386 (Scopus ID)
Projects
ELLIIT
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Note

open access

Available from: 2022-12-23 Created: 2022-12-23 Last updated: 2023-03-02Bibliographically approved
Minhas, N. M., Irshad, M., Petersen, K. & Börstler, J. (2023). Lessons learned from replicating a study on information-retrieval based test case prioritization. Software quality journal, 31(4), 1527-1559
Open this publication in new window or tab >>Lessons learned from replicating a study on information-retrieval based test case prioritization
2023 (English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 31, no 4, p. 1527-1559Article in journal (Refereed) Published
Abstract [en]

Replication studies help solidify and extend knowledge by evaluating previous studies’ findings. Software engineering literature showed that too few replications are conducted focusing on software artifacts without the involvement of humans. This study aims to replicate an artifact-based study on software testing to address the gap related to replications. In this investigation, we focus on (i) providing a step-by-step guide of the replication, reflecting on challenges when replicating artifact-based testing research and (ii) evaluating the replicated study concerning the validity and robustness of the findings. We replicate a test case prioritization technique proposed by Kwon et al. We replicated the original study using six software programs, four from the original study and two additional software programs. We automated the steps of the original study using a Jupyter notebook to support future replications. Various general factors facilitating replications are identified, such as (1) the importance of documentation; (2) the need for assistance from the original authors; (3) issues in the maintenance of open-source repositories (e.g., concerning needed software dependencies, versioning); and (4) availability of scripts. We also noted observations specific to the study and its context, such as insights from using different mutation tools and strategies for mutant generation. We conclude that the study by Kwon et al. is partially replicable for small software programs and could be automated to facilitate software practitioners, given the availability of required information. However, it is hard to implement the technique for large software programs with the current guidelines. Based on lessons learned, we suggest that the authors of original studies need to publish their data and experimental setup to support the external replications. © 2023, The Author(s).

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Replication, Regression testing, Technique, Test case prioritization, Information retrieval, SIR
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-23631 (URN)10.1007/s11219-023-09650-4 (DOI)001084224100001 ()2-s2.0-85174265778 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2022-09-13 Created: 2022-09-13 Last updated: 2023-12-05Bibliographically approved
Minhas, N. M., Koppula, T. R., Petersen, K. & Börstler, J. (2023). Using goal-question-metric to Compare Research and Practice Perspectives on Regression Testing. Journal of Software: Evolution and Process, 35(2), Article ID e2506.
Open this publication in new window or tab >>Using goal-question-metric to Compare Research and Practice Perspectives on Regression Testing
2023 (English)In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 35, no 2, article id e2506Article in journal (Refereed) Published
Abstract [en]

Regression testing is challenging because of its complexity and the amount of effort and time it requires, especially in large-scale environments with continuous integration and delivery. Regression test selection and prioritization techniques have been proposed in the literature to address the regression testing challenges, but adoption rates of these techniques in industry are not encouraging. One of the possible reasons could be the disparity in the regression testing goals in industry and literature. 

This work compares the research perspective to industry practice on regression testing goals, corresponding information needs, and metrics required to evaluate these goals. We have conducted a literature review of 44 research papers and a survey with 56 testing practitioners. The survey comprises 11 interviews and 45 responses to an online questionnaire. 

We identified that industry and research accentuate different regression testing goals. For instance, the literature emphasizes increasing the fault detection rates of test suites and early identification of critical faults. In contrast, the practitioners' focus is on test suite maintenance, controlled fault slippage, and awareness of changes. Similarly, the literature suggests maintaining information needs from test case execution histories to evaluate regression testing techniques based on various metrics, whereas, at large, the practitioners do not use the metrics suggested in the literature. 

To bridge the research and practice gap, based on the literature and survey findings, we have created a goal-question-metric (GQM) model that maps the regression testing goals, associated information needs, and metrics from both perspectives. The GQM model can guide researchers in proposing new techniques closer to industry contexts. Practitioners can benefit from information needs and metrics presented in the literature and can use GQM as a tool to follow their regression testing goals. 

Place, publisher, year, edition, pages
John Wiley & Sons, 2023
Keywords
Regression testing, Goals, Objectives, Measures, Metrics, GQM
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-23630 (URN)10.1002/smr.2506 (DOI)000852963100001 ()2-s2.0-85137875656 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2022-09-13 Created: 2022-09-13 Last updated: 2023-06-19Bibliographically approved
Salleh, N., Mendes, E., Mendes, F., Dissanayake Lekamlage, C. & Petersen, K. (2023). Value-based Software Engineering: A Systematic Mapping Study. e-Informatica Software Engineering Journal, 17(1), Article ID 230106.
Open this publication in new window or tab >>Value-based Software Engineering: A Systematic Mapping Study
Show others...
2023 (English)In: e-Informatica Software Engineering Journal, ISSN 1897-7979, E-ISSN 2084-4840, Vol. 17, no 1, article id 230106Article in journal (Refereed) Published
Abstract [en]

Background: Integrating value-oriented perspectives into the principles and practices of software engineering is fundamental to ensure that software development activities address key stakeholders’ views and also balance short-and long-term goals. This is put forward in the discipline of value-based software engineering (VBSE) Aim: This study aims to provide an overview of VBSE with respect to the research efforts that have been put into VBSE. Method: We conducted a systematic mapping study to classify evidence on value definitions, studies’ quality, VBSE principles and practices, research topics, methods, types, contribution facets, and publication venues. Results: From 143 studies we found that the term “value” has not been clearly defined in many studies. VB Requirements Engineering and VB Planning and Control were the two principles mostly investigated, whereas VB Risk Management and VB People Management were the least researched. Most studies showed very good reporting and relevance quality, acceptable credibility, but poor in rigour. Main research topic was Software Requirements and case study research was the method used the most. The majority of studies contribute towards methods and processes, while very few studies have proposed metrics and tools. Conclusion: We highlighted the research gaps and implications for research and practice to support VBSE. © 2023 The Authors.

Place, publisher, year, edition, pages
Wroclaw University of Technology, 2023
Keywords
Systematic mapping, value-based software engineering, VBSE, Mapping, Requirements engineering, Software design, Development activity, Long-term goals, Principles and practices, Research efforts, Research topics, Study qualities, Systematic mapping studies, Value-based software engineerings, Risk management
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-25210 (URN)10.37190/e-inf230106 (DOI)001094860300001 ()2-s2.0-85162861356 (Scopus ID)
Available from: 2023-08-06 Created: 2023-08-06 Last updated: 2023-12-04Bibliographically approved
Irshad, M., Börstler, J. & Petersen, K. (2022). Supporting Refactoring of BDD Specifications - An Empirical Study. Information and Software Technology, 141, Article ID 106717.
Open this publication in new window or tab >>Supporting Refactoring of BDD Specifications - An Empirical Study
2022 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 141, article id 106717Article in journal (Refereed) Published
Abstract [en]

Context: Behavior-driven development (BDD) is a variant of test-driven development where specifications are described in a structured domain-specific natural language. Although refactoring is a crucial activity of BDD, little research is available on the topic.

Objective: To support practitioners in refactoring BDD specifications by (1) proposing semi-automated approaches to identify refactoring candidates; (2) defining refactoring techniques for BDD specifications; and (3) evaluating the proposed identification approaches in an industry context.

Method: Using Action Research, we have developed an approach for identifying refactoring candidates in BDD specifications based on two measures of similarity and applied the approach in two projects of a large software organization. The accuracy of the measures for identifying refactoring candidates was then evaluated against an approach based on machine learning and a manual approach based on practitioner perception.

Results: We proposed two measures of similarity to support the identification of refactoring candidates in a BDD specification base; (1) normalized compression similarity (NCS) and (2) similarity ratio (SR). A semi-automated approach based on NCS and SR was developed and applied to two industrial cases to identify refactoring candidates. Our results show that our approach can identify candidates for refactoring 6o times faster than a manual approach. Our results furthermore showed that our measures accurately identified refactoring candidates compared with a manual identification by software practitioners and outperformed an ML-based text classification approach. We also described four types of refactoring techniques applicable to BDD specifications; merging candidates, restructuring candidates, deleting duplicates, and renaming specification titles. 

Conclusion: Our results show that NCS and SR can help practitioners in accurately identifying BDD specifications that are suitable candidates for refactoring, which also decreases the time for identifying refactoring candidates.

Place, publisher, year, edition, pages
Elsevier, 2022
Keywords
BDD, Behavior-driven development, Normalized Compression Distance (NCD), Normalized Compression Similarity (NCS), Refactoring, Reuse, Similarity ratio (SR), Specifications, Testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-21988 (URN)10.1016/j.infsof.2021.106717 (DOI)000701988700002 ()2-s2.0-85113689195 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Note

open access

Available from: 2021-07-11 Created: 2021-07-11 Last updated: 2023-06-07Bibliographically approved
Irshad, M. & Petersen, K. (2021). A systematic reuse process for automated acceptance tests: Construction and elementary evaluation. e-Informatica Software Engineering Journal, 15(1), 133-162
Open this publication in new window or tab >>A systematic reuse process for automated acceptance tests: Construction and elementary evaluation
2021 (English)In: e-Informatica Software Engineering Journal, ISSN 1897-7979, E-ISSN 2084-4840, Vol. 15, no 1, p. 133-162Article in journal (Refereed) Published
Abstract [en]

Context: Automated acceptance testing validates a product's functionality from the customer's perspective. Text-based automated acceptance tests (AATs) have gained popularity because they link requirements and testing.

Objective: To propose and evaluate a cost-effective systematic reuse process for automated acceptance tests.

Method: A systematic approach, method engineering, is used to construct a systematic reuse process for automated acceptance tests. The techniques to support searching, assessing, adapting the reusable tests are proposed and evaluated. The constructed process is evaluated using (i) qualitative feedback from software practitioners and (ii) a demonstration of the process in an industry setting. The process was evaluated for three constraints: performance expectancy, effort expectancy, and facilitating conditions.

Results: The process consists of eleven activities that support development for reuse, development with reuse, and assessment of the costs and benefits of reuse. During the evaluation, practitioners found the process a useful method to support reuse. In the industrial demonstration, it was noted that the activities in the solution helped in developing an automated acceptance test with reuse faster than creating a test from scratch i.e., searching, assessment, and adaptation parts.

Conclusion: The process is found to be useful and relevant to the industry during the preliminary investigation. 

Place, publisher, year, edition, pages
Wroclaw University of Technology, 2021
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-21989 (URN)10.37190/e-Inf210107 (DOI)000711485700001 ()
Note

open access

Available from: 2021-07-11 Created: 2021-07-11 Last updated: 2023-06-07Bibliographically approved
Irshad, M., Britto, R. & Petersen, K. (2021). Adapting Behavior Driven Development (BDD) for large-scale software systems. Journal of Systems and Software, 177, Article ID 110944.
Open this publication in new window or tab >>Adapting Behavior Driven Development (BDD) for large-scale software systems
2021 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 177, article id 110944Article in journal (Refereed) Published
Abstract [en]

Context: Large-scale software projects require interaction between many stakeholders. Behavior-driven development (BDD) facilitates collaboration between stakeholders, and an adapted BDD process can help improve cooperation in a large-scale project. Objective: The objective of this study is to propose and empirically evaluate a BDD based process adapted for large-scale projects. Method: A technology transfer model was used to propose a BDD based process for large-scale projects. We conducted six workshop sessions to understand the challenges and benefits of BDD. Later, an industrial evaluation was performed for the process with the help of practitioners. Results: From our investigations, understanding of a business aspect of requirements, their improved quality, a guide to system-level use-cases, reuse of artifacts, and help for test organization are found as benefits of BDD. Practitioners identified the following challenges: specification and ownership of behaviors, adoption of new tools, the software projects’ scale, and versioning of behaviors. We proposed a process to address these challenges and evaluated the process with the help of practitioners. Conclusion: The evaluation proved that BDD could be adapted and used to facilitate interaction in large-scale software projects in the software industry. The feedback from the practitioners helped in improving the proposed process. © 2021 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2021
Keywords
BDD, Behavior-driven, Large-scale, Software processes, System of systems, Boolean functions, Software engineering, Technology transfer, Business aspects, Industrial evaluations, Large-scale projects, Large-scale software projects, Large-scale software systems, Software industry, Software project, System levels, Large scale systems
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-21320 (URN)10.1016/j.jss.2021.110944 (DOI)000641355800005 ()2-s2.0-85103117256 (Scopus ID)
Note

open access

Available from: 2021-04-09 Created: 2021-04-09 Last updated: 2023-06-07Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-1532-8223

Search in DiVA

Show all publications