Change search
Link to record
Permanent link

Direct link
Publications (10 of 106) Show all publications
Iftikhar, U., Ali, N. b., Börstler, J. & Usman, M. (2024). A tertiary study on links between source code metrics and external quality attributes. Information and Software Technology, 165, Article ID 107348.
Open this publication in new window or tab >>A tertiary study on links between source code metrics and external quality attributes
2024 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 165, article id 107348Article, review/survey (Refereed) Published
Abstract [en]

Context: Several secondary studies have investigated the relationship between internal quality attributes, source code metrics and external quality attributes. Sometimes they have contradictory results. Objective: We synthesize evidence of the link between internal quality attributes, source code metrics and external quality attributes along with the efficacy of the prediction models used. Method: We conducted a tertiary review to identify, evaluate and synthesize secondary studies. We used several characteristics of secondary studies as indicators for the strength of evidence and considered them when synthesizing the results. Results: From 711 secondary studies, we identified 15 secondary studies that have investigated the link between source code and external quality. Our results show : (1) primarily, the focus has been on object-oriented systems, (2) maintainability and reliability are most often linked to internal quality attributes and source code metrics, with only one secondary study reporting evidence for security, (3) only a small set of complexity, coupling, and size-related source code metrics report a consistent positive link with maintainability and reliability, and (4) group method of data handling (GMDH) based prediction models have performed better than other prediction models for maintainability prediction. Conclusions: Based on our results, lines of code, coupling, complexity and the cohesion metrics from Chidamber & Kemerer (CK) metrics are good indicators of maintainability with consistent evidence from high and moderate-quality secondary studies. Similarly, four CK metrics related to coupling, complexity and cohesion are good indicators of reliability, while inheritance and certain cohesion metrics show no consistent evidence of links to maintainability and reliability. Further empirical studies are needed to explore the link between internal quality attributes, source code metrics and other external quality attributes, including functionality, portability, and usability. The results will help researchers and practitioners understand the body of knowledge on the subject and identify future research directions. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Code quality, Evidence, Product quality, Quality models, Tertiary review, Tertiary study, Codes (symbols), Computer programming languages, Data handling, Forecasting, Object oriented programming, Reliability, External quality, Internal quality, Products quality, Quality attributes, Quality modeling, Source code metrics, Maintainability
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-25555 (URN)10.1016/j.infsof.2023.107348 (DOI)001102357100001 ()2-s2.0-85174715019 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsKnowledge Foundation, 20190081
Available from: 2023-11-06 Created: 2023-11-06 Last updated: 2024-03-13Bibliographically approved
Iftikhar, U., Börstler, J., Ali, N. b. & Kopp, O. (2024). Identifying prevalent quality issues in code changes by analyzing reviewers' feedback.
Open this publication in new window or tab >>Identifying prevalent quality issues in code changes by analyzing reviewers' feedback
2024 (English)Manuscript (preprint) (Other academic)
Abstract [en]

Context: Code reviewers provide valuable feedback during the code review. Identifying common issues described in the reviewers' feedback can provide input for context-specific software improvement opportunities. However, the use of reviewer feedback for this purpose is currently less explored.

Objective: Assessing if and how automation can derive themes in reviewers' feedback and whether these themes help to identify recurring quality-related issues in code changes.

Method: We conducted a case study using the JabRef system to distinguish reviewers' feedback on merged and abandoned code changes for the analysis. We used topic modeling to identify themes in 5,560 code review comments. The resulting themes were analyzed and named by a domain expert from JabRef.

Results: The domain expert considered the identified themes from the proposed automation approach to represent quality-related issues. We found that different quality issues are pointed out in code reviews for merged and abandoned code changes. 

Conclusions: The results indicate the usefulness of our proposed automation approach in utilizing code review comments for understanding the prevalent code quality issues that can help derive targeted and context-bound improvement actions.

National Category
Computer Systems
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-25611 (URN)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2024-01-23 Created: 2024-01-23 Last updated: 2024-03-13Bibliographically approved
Iftikhar, U., Ali, N. b., Börstler, J. & Usman, M. (2023). A catalog of source code metrics – a tertiary study. In: Daniel Mendez, Dietmar Winkler, Johannes Kross, Stefan Biffl, Johannes Bergsmann (Ed.), Software Quality: Higher Software Quality through Zero Waste Development. Paper presented at 15th International Conference on Software Quality, SWQD 2023, Munich, Germany, May 23-25, 2023 (pp. 87-106). Springer, 472
Open this publication in new window or tab >>A catalog of source code metrics – a tertiary study
2023 (English)In: Software Quality: Higher Software Quality through Zero Waste Development / [ed] Daniel Mendez, Dietmar Winkler, Johannes Kross, Stefan Biffl, Johannes Bergsmann, Springer, 2023, Vol. 472, p. 87-106Conference paper, Published paper (Refereed)
Abstract [en]

Context: A large number of source code metrics are reported in the literature. It is necessary to systematically collect, describe and classify source code metrics to support research and practice.Objective: We aim to utilize existing secondary studies to develop a cat- alog of source code metrics together with their descriptions. The catalog will also provide information about which units of code (e.g., operators, operands, lines of code, variables, parameters, code blocks, or functions) are used to measure the internal quality attributes and the scope on which they are collected. 

Method: We conducted a tertiary study to identify secondary studies re- porting source code metrics. We have classified the source code metrics according to the measured internal quality attributes, the units of code used in the measures, and the scope at which the source code metrics are collected. 

Results: From 711 secondary studies, we identified 52 relevant secondary studies. We reported 423 source code metrics together with their de- scriptions and the internal quality attributes they measure. Source code metrics predominantly incorporate function as a unit of code to measure internal quality attributes. In contrast, several source code metrics use more than one unit of code when measuring internal quality attributes. Nearly 51% of the source code metrics are collected at the class scope, while almost 12% and 15% of source code metrics are collected at module and application levels, respectively. 

Conclusions: Researchers and practitioners can use the extensive catalog to assess which source code metrics meet their individual needs based on the description and classification scheme presented. 

Place, publisher, year, edition, pages
Springer, 2023
Series
Lecture Notes in Business Information Processing, ISSN 1865-1348, E-ISSN 1865-1356 ; 472
Keywords
Internal quality attributes, Code measurement, Code quality, Ter- tiary study, Source code metrics
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-24650 (URN)10.1007/978-3-031-31488-9_5 (DOI)2-s2.0-85161231906 (Scopus ID)978-3-031-31487-2 (ISBN)978-3-031-31488-9 (ISBN)
Conference
15th International Conference on Software Quality, SWQD 2023, Munich, Germany, May 23-25, 2023
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications, B07
Available from: 2023-05-30 Created: 2023-05-30 Last updated: 2024-03-13Bibliographically approved
Laiq, M., Ali, N. b., Börstler, J. & Engström, E. (2023). A data-driven approach for understanding invalid bug reports: An industrial case study. Information and Software Technology, 164, Article ID 107305.
Open this publication in new window or tab >>A data-driven approach for understanding invalid bug reports: An industrial case study
2023 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 164, article id 107305Article in journal (Refereed) Published
Abstract [en]

Context: Bug reports created during software development and maintenance do not always describe deviations from a system's valid behavior. Such invalid bug reports may consume significant resources and adversely affect the prioritization and resolution of valid bug reports. There is a need to identify preventive actions to reduce the inflow of invalid bug reports. Existing research has shown that manually analyzing invalid bug report descriptions provides cues regarding preventive actions. However, such a manual approach is not cost-effective due to the time required to analyze a sufficiently large number of bug reports needed to identify useful patterns. Furthermore, the analysis needs to be repeated as the underlying causes of invalid bug reports change over time. Objective: In this study, we propose and evaluate the use of Latent Dirichlet Allocation (LDA), a topic modeling approach, to support practitioners in suggesting preventive actions to avoid the creation of similar invalid bug reports in the future. Method: In an industrial case study, we first manually analyzed descriptions of invalid bug reports to identify common patterns in their descriptions. We further investigated to what extent LDA can support this manual process. We used expert-based validation to evaluate the relevance of identified common patterns and their usefulness in suggesting preventive measures. Results: We found that invalid bug reports have common patterns that are perceived as relevant, and they can be used to devise preventive measures. Furthermore, the identification of common patterns can be supported with automation. Conclusion: Using LDA, practitioners can effectively identify representative groups of bug reports (i.e., relevant common patterns) from a large number of bug reports and analyze them further to devise preventive measures. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Bug classification, Bug management, Invalid bug reports, LDA, Software analytics, Software maintenance, Topic modeling, Cost effectiveness, Software design, Statistics, Bug managements, Bug reports, Industrial case study, Invalid bug report, Latent Dirichlet allocation, Preventive action, Preventive measures, Software analytic, Computer software maintenance
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-25295 (URN)10.1016/j.infsof.2023.107305 (DOI)001053508400001 ()2-s2.0-85166970380 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsKnowledge Foundation, 20220235
Available from: 2023-08-18 Created: 2023-08-18 Last updated: 2023-09-08Bibliographically approved
Minhas, N. M., Börstler, J. & Petersen, K. (2023). Checklists to support decision-making in regression testing. Journal of Systems and Software, 202, Article ID 111697.
Open this publication in new window or tab >>Checklists to support decision-making in regression testing
2023 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 202, article id 111697Article in journal (Refereed) Published
Abstract [en]

Context: Practitioners working in large-scale software development face many challenges in regression testing activities. One of the reasons is the lack of a structured regression testing process. In this regard, checklists can help practitioners keep track of essential regression testing activities and add structure to the regression testing process to a certain extent. Objective: This study aims to introduce regression testing checklists so test managers/teams can use them: (1) to assess whether test teams/members are ready to begin regression testing, and (2) to keep track of essential regression testing activities while planning and executing regression tests. Method: We used interviews, workshops, and questionnaires to design, evolve, and evaluate regression testing checklists. In total, 25 practitioners from 12 companies participated in creating the checklist. Twenty-three of them participated in checklists evolution and evaluation. Results: We identified activities practitioners consider significant while planning, performing, and analyzing regression testing. We designed regression testing checklists based on these activities to help practitioners make informed decisions during regression testing. With the help of practitioners, we evolved these checklists into two iterations. Finally, the practitioners provided feedback on the proposed checklists. All respondents think the proposed checklists are useful and customizable for their environments, and 80% think checklists cover aspects essential for regression testing. Conclusion: The proposed regression testing checklists can be useful for test managers to assess their team/team members’ readiness and decide when to start and stop regression testing. The checklists can be used to record the steps required while planning and executing regression testing. Further, these checklists can provide a basis for structuring the regression testing process in varying contexts. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Regression testing, Checklists, Test manager, Team readiness, Process improvement
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-23675 (URN)10.1016/j.jss.2023.111697 (DOI)000989289800001 ()2-s2.0-85153245617 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2022-09-18 Created: 2022-09-18 Last updated: 2023-06-12Bibliographically approved
Börstler, J., Bennin, K. E., Hooshangi, S., Jeuring, J., Keuning, H., Kleiner, C., . . . van Assema, J. (2023). Developers talking about code quality. Empirical Software Engineering, 28(6), Article ID 128.
Open this publication in new window or tab >>Developers talking about code quality
Show others...
2023 (English)In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 28, no 6, article id 128Article in journal (Refereed) Published
Abstract [en]

There are many aspects of code quality, some of which are difficult to capture or to measure. Despite the importance of software quality, there is a lack of commonly accepted measures or indicators for code quality that can be linked to quality attributes. We investigate software developers’ perceptions of source code quality and the practices they recommend to achieve these qualities. We analyze data from semi-structured interviews with 34 professional software developers, programming teachers and students from Europe and the U.S. For the interviews, participants were asked to bring code examples to exemplify what they consider good and bad code, respectively. Readability and structure were used most commonly as defining properties for quality code. Together with documentation, they were also suggested as the most common target properties for quality improvement. When discussing actual code, developers focused on structure, comprehensibility and readability as quality properties. When analyzing relationships between properties, the most commonly talked about target property was comprehensibility. Documentation, structure and readability were named most frequently as source properties to achieve good comprehensibility. Some of the most important source code properties contributing to code quality as perceived by developers lack clear definitions and are difficult to capture. More research is therefore necessary to measure the structure, comprehensibility and readability of code in ways that matter for developers and to relate these measures of code structure, comprehensibility and readability to common software quality attributes.

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Code quality, Semi-structured interviews, Software development, Source code properties, Quality perception
National Category
Software Engineering
Research subject
Software Engineering; Computer Science
Identifiers
urn:nbn:se:bth-25436 (URN)10.1007/s10664-023-10381-0 (DOI)001073677200001 ()2-s2.0-85171886124 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2023-10-05 Created: 2023-10-05 Last updated: 2024-01-02Bibliographically approved
Börstler, J., Ali, N. b. & Petersen, K. (2023). Double-counting in software engineering tertiary studies — An overlooked threat to validity. Information and Software Technology, 158, Article ID 107174.
Open this publication in new window or tab >>Double-counting in software engineering tertiary studies — An overlooked threat to validity
2023 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 158, article id 107174Article, review/survey (Refereed) Published
Abstract [en]

Context: Double-counting in a literature review occurs when the same data, population, or evidence is erroneously counted multiple times during synthesis. Detecting and mitigating the threat of double-counting is particularly challenging in tertiary studies. Although this topic has received much attention in the health sciences, it seems to have been overlooked in software engineering. Objective: We describe issues with double-counting in tertiary studies, investigate the prevalence of the issue in software engineering, and propose ways to identify and address the issue. Method: We analyze 47 tertiary studies in software engineering to investigate in which ways they address double-counting and whether double-counting might be a threat to validity in them. Results: In 19 of the 47 tertiary studies, double-counting might bias their results. Of those 19 tertiary studies, only 5 consider double-counting a threat to their validity, and 7 suggest strategies to address the issue. Overall, only 9 of the 47 tertiary studies, acknowledge double-counting as a potential general threat to validity for tertiary studies. Conclusions: Double-counting is an overlooked issue in tertiary studies in software engineering, and existing design and evaluation guidelines do not address it sufficiently. Therefore, we propose recommendations that may help to identify and mitigate double-counting in tertiary studies. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Population statistics, Bias, Double counting, Empirical, Guideline, Meta-review, Overview of review, Recommendation, Research method, Review of review, Tertiary review, Tertiary study, Umbrella review, Software engineering, Double-counting, Guidelines, Overview of reviews, Recommendations, Research methods, Review of reviews
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-24419 (URN)10.1016/j.infsof.2023.107174 (DOI)001005614800001 ()2-s2.0-85150795598 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2023-04-07 Created: 2023-04-07 Last updated: 2023-06-30Bibliographically approved
Börstler, J., Ali, N. b., Svensson, M. & Petersen, K. (2023). Investigating Acceptance Behavior in Software Engineering – Theoretical Perspectives. Journal of Systems and Software, 198, Article ID 111592.
Open this publication in new window or tab >>Investigating Acceptance Behavior in Software Engineering – Theoretical Perspectives
2023 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 198, article id 111592Article in journal (Refereed) Published
Abstract [en]

Background: Software engineering research aims to establish software development practice on a scientific basis. However, the evidence of the efficacy of technology is insufficient to ensure its uptake in industry. In the absence of a theoretical frame of reference, we mainly rely on best practices and expert judgment from industry-academia collaboration and software process improvement research to improve the acceptance of the proposed technology. Objective: To identify acceptance models and theories and discuss their applicability in the research of acceptance behavior related to software development.Method: We analyzed literature reviews within an interdisciplinary team to identify models and theories relevant to software engineering research. We further discuss acceptance behavior from the human information processing perspective of automatic and affect-driven processes (“fast” system 1 thinking) and rational and rule-governed processes (“slow” system 2 thinking). Results: We identified 30 potentially relevant models and theories. Several of them have been used in researching acceptance behavior in contexts related to software development, but few have been validated in such contexts. They use constructs that capture aspects of (automatic) system 1 and (rational) system 2 oriented processes. However, their operationalizations focus on system 2-oriented processes indicating a rational view of behavior, thus overlooking important psychological processes underpinning behavior. Conclusions: Software engineering research may use acceptance behavior models and theories more extensively to understand and predict practice adoption in the industry. Such theoretical foundations will help improve the impact of software engineering research. However, more consideration should be given to their validation, overlap, construct operationalization, and employed data collection mechanisms when using these models and theories.

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Acceptance behavior, dual process theory, technology acceptance, theory, TAM, UTAUT, TPB
National Category
Software Engineering Psychology
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-24132 (URN)10.1016/j.jss.2022.111592 (DOI)000915632900001 ()2-s2.0-85146227386 (Scopus ID)
Projects
ELLIIT
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Note

open access

Available from: 2022-12-23 Created: 2022-12-23 Last updated: 2023-03-02Bibliographically approved
Minhas, N. M., Irshad, M., Petersen, K. & Börstler, J. (2023). Lessons learned from replicating a study on information-retrieval based test case prioritization. Software quality journal, 31(4), 1527-1559
Open this publication in new window or tab >>Lessons learned from replicating a study on information-retrieval based test case prioritization
2023 (English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 31, no 4, p. 1527-1559Article in journal (Refereed) Published
Abstract [en]

Replication studies help solidify and extend knowledge by evaluating previous studies’ findings. Software engineering literature showed that too few replications are conducted focusing on software artifacts without the involvement of humans. This study aims to replicate an artifact-based study on software testing to address the gap related to replications. In this investigation, we focus on (i) providing a step-by-step guide of the replication, reflecting on challenges when replicating artifact-based testing research and (ii) evaluating the replicated study concerning the validity and robustness of the findings. We replicate a test case prioritization technique proposed by Kwon et al. We replicated the original study using six software programs, four from the original study and two additional software programs. We automated the steps of the original study using a Jupyter notebook to support future replications. Various general factors facilitating replications are identified, such as (1) the importance of documentation; (2) the need for assistance from the original authors; (3) issues in the maintenance of open-source repositories (e.g., concerning needed software dependencies, versioning); and (4) availability of scripts. We also noted observations specific to the study and its context, such as insights from using different mutation tools and strategies for mutant generation. We conclude that the study by Kwon et al. is partially replicable for small software programs and could be automated to facilitate software practitioners, given the availability of required information. However, it is hard to implement the technique for large software programs with the current guidelines. Based on lessons learned, we suggest that the authors of original studies need to publish their data and experimental setup to support the external replications. © 2023, The Author(s).

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Replication, Regression testing, Technique, Test case prioritization, Information retrieval, SIR
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-23631 (URN)10.1007/s11219-023-09650-4 (DOI)001084224100001 ()2-s2.0-85174265778 (Scopus ID)
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2022-09-13 Created: 2022-09-13 Last updated: 2023-12-05Bibliographically approved
Iftikhar, U., Börstler, J. & Ali, N. b. (2023). On potential improvements in the analysis of the evolution of themes in code review comments. In: Proceedings - 2023 49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023: . Paper presented at 49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023, Durres, Sept. 6th – Sept. 8th, 2023 (pp. 340-347). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>On potential improvements in the analysis of the evolution of themes in code review comments
2023 (English)In: Proceedings - 2023 49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 340-347Conference paper, Published paper (Refereed)
Abstract [en]

Context: The modern code review process is considered an essential quality assurance step in software development. The code review comments generated can provide insights regarding source code quality and development practices. However, the large number of code review comments makes it challenging to identify interesting patterns manually. In a recent study, Wen et al. used traditional topic modeling to analyze the evolution of code review comments. Their approach could identify interesting patterns that may lead to improved development practices.Objective: In this study, we investigate potential improvements to Wen et al.'s state-of-the-art approach to analyze the evolution of code review comments.Method: We used 209,166 code review comments from three open-source systems to explore and empirically analyze alternative design and implementation choices and demonstrate their impact.Results: We identified the following potential improvements to the current state-of-the-art as described by Wen et al.: 1) utilize a topic modeling method that is optimized for short texts, 2) a refined approach for identifying a suitable number of topics, and 3) a more elaborate approach for analyzing topic evolution. Our results indicate that the proposed changes have quantitatively different results than the current approach. The qualitative interpretation of the topics generated with our changes indicates their usefulness.Conclusions: Our results indicate the potential usefulness of changes to state-of-the-art approaches to analyzing the evolution of code review comments, with practical implications for researchers and practitioners. However, further research is required to compare the effectiveness of both approaches. © 2023 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
Modern Code Reviews, Source code quality, NLP
National Category
Computer Systems
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-25598 (URN)10.1109/SEAA60479.2023.00059 (DOI)2-s2.0-85183313412 (Scopus ID)9798350342352 (ISBN)
Conference
49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023, Durres, Sept. 6th – Sept. 8th, 2023
Projects
ELLIIT, the Strategic Research Area within IT and Mobile Communications,
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2023-11-12 Created: 2023-11-12 Last updated: 2024-03-13Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-0639-4234

Search in DiVA

Show all publications