How Many Papers Should You Review?: A Research Synthesis of Systematic Literature Reviews in Software Engineering
2023 (English)In: International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, 2023Conference paper, Published paper (Refereed)
Abstract [en]
[Context] Systematic Literature Review (SLR) has been a major type of study published in Software Engineering (SE) venues for about two decades. However, there is a lack of understanding of whether an SLR is really needed in comparison to a more conventional literature review. Very often, SE researchers embark on an SLR with such doubts. We aspire to provide more understanding of when an SLR in SE should be conducted. [Objective] The first step of our investigation was focused on the dataset, i.e., the reviewed papers, in an SLR, which indicates the development of a research topic or area. The objective of this step is to provide a better understanding of the characteristics of the datasets of SLRs in SE. [Method] A research synthesis was conducted on a sample of 170 SLRs published in top-tier SE journals. We extracted and analysed the quantitative attributes of the datasets of these SLRs. [Results] The findings show that the median size of the datasets in our sample is 57 reviewed papers, and the median review period covered is 14 years. The number of reviewed papers and review period have a very weak and non-significant positive correlation. [Conclusions] The results of our study can be used by SE researchers as an indicator or benchmark to understand whether an SLR is conducted at a good time. © 2023 IEEE.
Place, publisher, year, edition, pages
IEEE Computer Society, 2023.
Series
International Symposium on Empirical Software Engineering and Measurement, ISSN 1949-3770, E-ISSN 1949-3789
Keywords [en]
Methodological Study, Research Synthesis, SLR, Software Engineering, Systematic Literature Review, Paper, Literature reviews, Methodological studies, Positive correlations, Quantitative attributes, Research areas, Research topics, Software engineering journals
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-25783DOI: 10.1109/ESEM56168.2023.10304863Scopus ID: 2-s2.0-85178657649ISBN: 9781665452236 (print)OAI: oai:DiVA.org:bth-25783DiVA, id: diva2:1819848
Conference
17th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2023, New Orleans, 26 October through 27 October 2023
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications2023-12-152023-12-152023-12-15Bibliographically approved