Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Guidelines for including grey literature and conducting multivocal literature reviews in software engineering
Wageningen University, NLD.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0003-3818-4442
University of Oulu, FIN.
2019 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 106, p. 101-121Article in journal (Refereed) Published
Abstract [en]

Context: A Multivocal Literature Review (MLR) is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts, videos and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). MLRs are useful for both researchers and practitioners since they provide summaries both the state-of-the art and –practice in a given area. MLRs are popular in other fields and have recently started to appear in software engineering (SE). As more MLR studies are conducted and reported, it is important to have a set of guidelines to ensure high quality of MLR processes and their results. Objective: There are several guidelines to conduct SLR studies in SE. However, several phases of MLRs differ from those of traditional SLRs, for instance with respect to the search process and source quality assessment. Therefore, SLR guidelines are only partially useful for conducting MLR studies. Our goal in this paper is to present guidelines on how to conduct MLR studies in SE. Method: To develop the MLR guidelines, we benefit from several inputs: (1) existing SLR guidelines in SE, (2), a literature survey of MLR guidelines and experience papers in other fields, and (3) our own experiences in conducting several MLRs in SE. We took the popular SLR guidelines of Kitchenham and Charters as the baseline and extended/adopted them to conduct MLR studies in SE. All derived guidelines are discussed in the context of an already-published MLR in SE as the running example. Results: The resulting guidelines cover all phases of conducting and reporting MLRs in SE from the planning phase, over conducting the review to the final reporting of the review. In particular, we believe that incorporating and adopting a vast set of experience-based recommendations from MLR guidelines and experience papers in other fields have enabled us to propose a set of guidelines with solid foundations. Conclusion: Having been developed on the basis of several types of experience and evidence, the provided MLR guidelines will support researchers to effectively and efficiently conduct new MLRs in any area of SE. The authors recommend the researchers to utilize these guidelines in their MLR studies and then share their lessons learned and experiences. © 2018

Place, publisher, year, edition, pages
Elsevier B.V. , 2019. Vol. 106, p. 101-121
Keywords [en]
Evidence-based software engineering, Grey literature, Guidelines, Literature study, Multivocal literature review, Systematic literature review, Systematic mapping study, Information systems, Evidence Based Software Engineering, Literature reviews, Literature studies, Systematic mapping studies, Software engineering
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-17110DOI: 10.1016/j.infsof.2018.09.006ISI: 000454381300006OAI: oai:DiVA.org:bth-17110DiVA, id: diva2:1255178
Available from: 2018-10-11 Created: 2018-10-11 Last updated: 2022-05-25Bibliographically approved

Open Access in DiVA

fulltext(772 kB)432 downloads
File information
File name FULLTEXT01.pdfFile size 772 kBChecksum SHA-512
0223260841d801add70e5b5193fc79f02baac5848cc33f621b04c8e21f3f81316354370fdc147546d3e6b5f6322797707a18d3512340f8d83a172728fef4c541
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Authority records

Felderer, Michael

Search in DiVA

By author/editor
Felderer, Michael
By organisation
Department of Software Engineering
In the same journal
Information and Software Technology
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 432 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 609 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf