Optimizing Startup Software Product Development through Generative AI: Investigating the value of Active Persona-generated feedback in assessing software quality and user alignment
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE credits
Student thesis
Abstract [en]
Background. The concept of personas for user simulation, introduced by Cooper in 1999, aimed to improve software design by providing detailed user representations. This addressed the issue of engineers designing for abstract "elastic users," leading to poor usability despite technical functionality. Pruitt and Adlin later refined this by emphasizing data-driven profiles capturing behavioral patterns. However, Pruitt and Grudin, followed by Chapman and Milham, highlighted implementation challenges and the "representational paradox"—the difficulty in proving persona validity. The static nature of traditional personas became problematic with rapid technological advancements, as noted by Salminen et al. Despite drawbacks, personas remained valuable for humanizing design. AI research between 2015 and 2020 explored AI-driven personas for more accurate, data-driven user representations. The rise of generative AI further increased interest in AI's potential to create dynamic user personas, though careful validation is necessary due to potential unreliability.
Objectives. This research aims to validate the concept of Active Personas (APs) as an effective tool for generating contextual software development feedback. By systematically analyzing the efficiency of AI-generated Active Personas, the study will thoroughly examine their ability to produce meaningful feedback that matches or exceeds real user feedback.
Methods. The research project will follow the Design Science Research (DSR) methodology to create Active Personas as artefacts with the objective of generating contextual feedback on selected codebases. By choosing the DSR framework as the research methodology, the aim is to create practical artefacts that address real-world challenges in the product development cycle.
Results. The study demonstrated that Active Personas (APs) powered by generative AI can efficiently implement diverse user perspectives based on personas and generate contextual feedback aligned with them and established usability principles. GPT-4o showed the fastest execution for image-based UI tasks (1m 14s), while LLaMA3 was the most efficient for code analysis (30m 12s). The feedback generated by APs with GPT-4 strongly aligned with usability standards and often matched real user feedback. In cases like readability and character display integrity, APs' insights not only validated user concerns but also provided deeper analysis and improvement suggestions. In OS-specific issues and codebase analysis, APs offered more generic and speculative technical analysis. APs that use a codebase analysis approach tend to be less efficient, context-aligned, and trustworthy.
Conclusions. APs proved to be a valuable tool for early-stage usability evaluation, offering efficiency and depth while highlighting certain gaps in real-world alignment. The application of APs in a startup environment can bring value to practitioners in the product development cycle, but human supervision in the evaluation process remains a key factor in ensuring contextual accuracy and mitigating potential AP-generated feedback inaccuracy.
Place, publisher, year, edition, pages
2025. , p. 52
Keywords [en]
Generative AI, Persona, Feedback, Usability, User-centered design
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:bth-28364OAI: oai:DiVA.org:bth-28364DiVA, id: diva2:1984403
Subject / course
PA2534 Master's Thesis (120 credits) in Software Engineering
Educational program
PAASW Master's Programme in Software Engineering 120,0 hp
Supervisors
Examiners
2025-08-062025-07-162025-09-30Bibliographically approved