Another Systematic Review? A Critical Analysis of Systematic Literature Reviews on Agile Effort and Cost Estimation
2025 (English)In: International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, 2025, p. 23-32Conference paper, Published paper (Refereed)
Abstract [en]
Background: Systematic literature reviews (SLRs) have become prevalent in software engineering research. Several researchers may conduct SLRs on similar topics without a prospective register for SLR protocols. However, even ignoring these unavoidable duplications of effort in the simultaneous conduct of SLRs, the proliferation of overlapping and often repetitive SLRs indicates that researchers are not extensively checking for existing SLRs on a topic. Given how effort-intensive it is to design, conduct, and report an SLR, the situation is less than ideal for software engineering research.
Aim: To understand how authors justify additional SLRs on a topic.
Method: To illustrate the issue and develop suggestions for improvement to address this issue, we have intentionally picked a sufficiently narrow but well-researched topic, i.e., effort estimation in Agile software development. We identify common justification patterns through a qualitative content analysis of 18 published SLRs. We further consider the citation data, publication years, publication venues, and the quality of the SLRs when interpreting the results.
Results: The common justification patterns include authors claiming gaps in coverage, methodological limitations in prior studies, temporal obsolescence of previous SLRs, or rapid technological and methodological advancements necessitating updated syntheses.
Conclusion: Our in-depth analysis of SLRs on a fairly narrow topic provides insights into SLRs in software engineering in general. By emphasizing the need for identifying existing SLRs and for justifying the undertaking of further SLRs, both in design and review guidelines and as a policy of conferences and journals, we can reduce the likelihood of duplication of effort and increase the rate of progress in the field.
Place, publisher, year, edition, pages
IEEE Computer Society, 2025. p. 23-32
Series
International Symposium on Empirical Software Engineering and Measurement, ISSN 1949-3770, E-ISSN 1949-3789
Keywords [en]
agile, cost estimation, effort estimation, need for a review, systematic reviews, Agile manufacturing systems, Cost benefit analysis, Cost engineering, Engineering research, Obsolescence, Publishing, Software design, Agile software development, Cost estimations, Critical analysis, Prospectives, Software engineering research, Systematic literature review, Systematic Review, Cost estimating
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-29289DOI: 10.1109/ESEM64174.2025.00051Scopus ID: 2-s2.0-105032704584ISBN: 9798331591472 (print)OAI: oai:DiVA.org:bth-29289DiVA, id: diva2:2049115
Conference
2025 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2025, Honolulu, Oct 2-3, 2025
Part of project
GIST – Gaining actionable Insights from Software Testing, Knowledge Foundation
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsKnowledge Foundation, 202202352026-03-272026-03-272026-03-31Bibliographically approved