Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Reporting case studies in systematic literature studies—An evidential problem
Queen's University Belfast, United Kingdom.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0003-0460-5253
2024 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 174, article id 107501Article in journal (Refereed) Published
Abstract [en]

Context: The term and label, “case study”, is not used consistently by authors of primary studies in software engineering research. It is not clear whether this problem also occurs for systematic literature studies (SLSs).

Objective: To investigate the extent to which SLSs in/correctly use the term and label, “case study”, when classifying primary studies.

Methods: We systematically collect two sub-samples (2010–2021 & 2022) comprising a total of eleven SLSs and 79 primary studies. We examine the designs of these SLSs, and then analyse whether the SLS authors and the primary-study authors correctly label the respective primary study as a “case study”.

Results: 76% of the 79 primary studies are misclassified by SLSs (with the two sub-samples having 60% and 81% misclassification, respectively). For 39% of the 79 studies, the SLSs propagate a mislabelling by the original authors, whilst for 37%, the SLSs introduce a new mislabel, thus making the problem worse. SLSs rarely present explicit definitions for “case study” and when they do, the definition is not consistent with established definitions.

Conclusions: SLSs are both propagating and exacerbating the problem of the mislabelling of primary studies as “case studies”, rather than – as we should expect of SLSs – correcting the labelling of primary studies, and thus improving the body of credible evidence. Propagating and exacerbating mislabelling undermines the credibility of evidence in terms of its quantity, quality and relevance to both practice and research. © 2024 The Author(s)

Place, publisher, year, edition, pages
Elsevier, 2024. Vol. 174, article id 107501
Keywords [en]
Case study, Credible evidence, Systematic literature review, Systematic mapping study, Systematic review, Case-studies, Labelings, Literature studies, Misclassifications, Software engineering research, Sub-samples, Systematic mapping studies, Software engineering
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-26541DOI: 10.1016/j.infsof.2024.107501ISI: 001252262000001Scopus ID: 2-s2.0-85195473840OAI: oai:DiVA.org:bth-26541DiVA, id: diva2:1876166
Available from: 2024-06-24 Created: 2024-06-24 Last updated: 2024-08-05Bibliographically approved

Open Access in DiVA

fulltext(688 kB)54 downloads
File information
File name FULLTEXT01.pdfFile size 688 kBChecksum SHA-512
b43df0833893503626513d27ad9da857011c4770efbc7d60352ad6ea5ce59d6c16b5eaeafb8f687a01af0080898c97c6ee807cd65e1086dd677a7d1e7f5f7d10
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Wohlin, Claes

Search in DiVA

By author/editor
Wohlin, Claes
By organisation
Department of Software Engineering
In the same journal
Information and Software Technology
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 55 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 364 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf