Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Double-counting in software engineering tertiary studies — An overlooked threat to validity
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0003-0639-4234
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-7266-5632
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. University of Applied Sciences, Germany.ORCID iD: 0000-0002-1532-8223
2023 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 158, article id 107174Article, review/survey (Refereed) Published
Abstract [en]

Context: Double-counting in a literature review occurs when the same data, population, or evidence is erroneously counted multiple times during synthesis. Detecting and mitigating the threat of double-counting is particularly challenging in tertiary studies. Although this topic has received much attention in the health sciences, it seems to have been overlooked in software engineering. Objective: We describe issues with double-counting in tertiary studies, investigate the prevalence of the issue in software engineering, and propose ways to identify and address the issue. Method: We analyze 47 tertiary studies in software engineering to investigate in which ways they address double-counting and whether double-counting might be a threat to validity in them. Results: In 19 of the 47 tertiary studies, double-counting might bias their results. Of those 19 tertiary studies, only 5 consider double-counting a threat to their validity, and 7 suggest strategies to address the issue. Overall, only 9 of the 47 tertiary studies, acknowledge double-counting as a potential general threat to validity for tertiary studies. Conclusions: Double-counting is an overlooked issue in tertiary studies in software engineering, and existing design and evaluation guidelines do not address it sufficiently. Therefore, we propose recommendations that may help to identify and mitigate double-counting in tertiary studies. © 2023 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2023. Vol. 158, article id 107174
Keywords [en]
Population statistics, Bias, Double counting, Empirical, Guideline, Meta-review, Overview of review, Recommendation, Research method, Review of review, Tertiary review, Tertiary study, Umbrella review, Software engineering, Double-counting, Guidelines, Overview of reviews, Recommendations, Research methods, Review of reviews
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-24419DOI: 10.1016/j.infsof.2023.107174ISI: 001005614800001Scopus ID: 2-s2.0-85150795598OAI: oai:DiVA.org:bth-24419DiVA, id: diva2:1749513
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsAvailable from: 2023-04-07 Created: 2023-04-07 Last updated: 2023-06-30Bibliographically approved

Open Access in DiVA

fulltext(831 kB)145 downloads
File information
File name FULLTEXT01.pdfFile size 831 kBChecksum SHA-512
587b2895dc73947775abc02cc2c90f498c4fd3e8a101fc2815b2f7d9397b9828faa2f406848be102b12804282d784a1d1952142a81513020d22c06b5bf7c3bb6
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Börstler, JürgenAli, Nauman binPetersen, Kai

Search in DiVA

By author/editor
Börstler, JürgenAli, Nauman binPetersen, Kai
By organisation
Department of Software Engineering
In the same journal
Information and Software Technology
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 145 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 89 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf