The applicability of Generative AI in Systematic Literature Reviews: Exploring GPT-4's Role in Automating and Assisting Researchers
2025 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE credits
Student thesis
Abstract [en]
This thesis examines the use of Large Language Models (LLMs) to automate key phases of the Systematic Literature Review (SLR) in Software Engineering (SE). Using qualitative interviews and a blind test, we assess the applicability, opportunities, and limitations of LLMs in research workflows.
Study selection (screening) and research identification (search) emerge as the most automatable steps. Repetitive screening is ideal for automation and keyword generation shows potential. However, limitations such as restricted database access and search strategy constraints hinder full automation. Other SLR steps are less suitable for automation. LLMs can reduce human bias, assist in screening, and handle tasks such as formatting, grammar checking, and summarizing.
Despite these benefits, there are concerns about LLM biases, transparency, and ethical issues concerning data privacy. Some question whether automating SLRs supports the fundamental goal of researcher learning. LLM-generated search strings are similar in quality to human-created ones but require manual adjustments for Boolean logic and formatting.
Although LLMs can help, they should not replace human oversight. Cautious automation can enhance, but not replace, traditional research methods. More research is needed to refine the use of LLM in SLRs, focusing on transparency, reliability, and ethics.
Place, publisher, year, edition, pages
2025. , p. 37
Keywords [en]
Systematic Literature Review, Automation, Validity Threats, GPT-4, LLMs
National Category
Robotics and automation
Identifiers
URN: urn:nbn:se:bth-27492OAI: oai:DiVA.org:bth-27492DiVA, id: diva2:1941180
Subject / course
PA1445 Kandidatkurs i Programvaruteknik; PA1445 Kandidatkurs i Programvaruteknik
Educational program
PAGPT Software Engineering; PAGWE Web Programming
Presentation
2025-01-16, G313ALC, Valhallavägen 1, Karlskrona, 12:30 (Swedish)
Supervisors
Examiners
2025-03-032025-02-272025-09-30Bibliographically approved