Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
On researcher bias in Software Engineering experiments
University of Bari, ITA.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0002-0679-4361
University of Basilicata, ITA.
University of Bari, ITA.
Show others and affiliations
2021 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 182, article id 111068Article in journal (Refereed) Published
Abstract [en]

Researcher bias occurs when researchers influence the results of an empirical study based on their expectations, either consciously or unconsciously. Researcher bias might be due to the use of Questionable Research Practices (QRPs). In research fields like medicine, blinding techniques have been applied to counteract researcher bias. In this paper, we present two studies to increase our body of knowledge on researcher bias in Software Engineering (SE) experiments, including: (i) QRPs potentially leading to researcher bias; (ii) causes behind researcher bias; and (iii) possible actions to counteract researcher bias with a focus on, but not limited to, blinding techniques. The former is an interview study, intended as an exploratory study, with nine experts of the empirical SE community. The latter is a quantitative survey with 51 respondents, who were experts of the above-mentioned community. The findings from the exploratory study represented the starting point to design the survey. In particular, we defined the questionnaire of this survey to support the findings from the exploratory study. From the interview study, it emerged that some QRPs (e.g., post-hoc outlier criteria) are acceptable in certain cases. Also, it appears that researcher bias is perceived in SE and, to counteract researcher bias, a number of solutions have been highlighted. For example, duplicating the data analysis in SE experiments or fostering open data policies in SE conferences/journals. The findings from the interview study are mostly confirmed by those from the survey, and allowed us to delineate recommendations to counteract researcher bias in SE experiments. Some recommendations are intended for SE researchers, while others are purposeful for the boards of SE research venues. © 2021 Elsevier Inc.

Place, publisher, year, edition, pages
Elsevier Inc. , 2021. Vol. 182, article id 111068
Keywords [en]
Blinding, Experimenter bias, Researcher bias, Survey, Open Data, Software engineering, Blinding technique, Body of knowledge, Empirical studies, Exploratory studies, Interview study, Research fields, Software engineering experiments, Surveys
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-22141DOI: 10.1016/j.jss.2021.111068ISI: 000697027700003Scopus ID: 2-s2.0-85114408763OAI: oai:DiVA.org:bth-22141DiVA, id: diva2:1595132
Available from: 2021-09-17 Created: 2021-09-17 Last updated: 2022-01-17Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Fucci, Davide

Search in DiVA

By author/editor
Fucci, Davide
By organisation
Department of Software Engineering
In the same journal
Journal of Systems and Software
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 28 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf