Automated testing, particularly for GUI-based systems, remains a costly and labor-intensive process and prone to errors. Despite advancements in automation, manual testing still dominates in industrial practice, resulting in delays, higher costs, and increased error rates. Large Language Models (LLMs) have shown great potential to automate tasks traditionally requiring human intervention, leveraging their cognitive-like abilities for test generation and evaluation. In this study, we present PathFinder, a Multi-Agent LLM (MALLM) framework that incorporates four agents responsible for (a) perception and summarization, (b) decision-making, (c) input handling and extraction, and (d) validation, which work collaboratively to automate exploratory web-based GUI testing. The goal of this study is to assess how different LLMs, applied to different agents, affect the efficacy of automated exploratory GUI testing. We evaluate PathFinder with three models, Mistral-Nemo, Gemma2, and Llama3.1, on four e-commerce websites. Thus, 27 permutations of the LLMs, across three agents (excluding the validation agent), to test the hypothesis that a solution with multiple agents, each using different LLMs, is more efficacious (efficient and effective) than a multi-agent solution where all agents use the same LLM. The results indicate that the choice of LLM constellation (combination of LLMs) significantly impacts efficacy, suggesting that a single LLM across agents may yield the best balance of efficacy (measured by F1-score). Hypothesis to explain this result include, but are not limited to: improved decision-making consistency and reduced task coordination discrepancies. The contributions of this study are an architecture for MALLM-based GUI testing, empirical results on its performance, and novel insights into how LLM selection impacts the efficacy of automated testing. 

In the evolving landscape of mobile applications, effective and efficient testing methods are crucial for ensuring high-quality user experiences. This paper introduces a novel end-To-end mobile testing technique designed to enhance exploratory testing by incorporating gamification strategies. We developed a plugin that integrates these innovative techniques, aiming to make the testing process more engaging and effective. With the use of a live feedback system, the plugin drives testers to thoroughly explore the application, leading to the discovery of more defects and improved software quality. Preliminary evaluation suggests that this approach could not only increase tester engagement but also improve the detection rate of critical issues. This research highlights the potential of merging exploratory testing with gamification, setting the stage for more dynamic and productive mobile testing methodologies. © 2024 Owner/Author.