Change search
Link to record
Permanent link

Direct link
BETA
Poulding, Simon
Alternative names
Publications (10 of 13) Show all publications
Irshad, M., Petersen, K. & Poulding, S. (2018). A systematic literature review of software requirements reuse approaches. Information and Software Technology, 93(Jan), 223-245
Open this publication in new window or tab >>A systematic literature review of software requirements reuse approaches
2018 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 93, no Jan, p. 223-245Article in journal (Refereed) Published
Abstract [en]

Context: Early software reuse is considered as the most beneficial form of software reuse. Hence, previous research has focused on supporting the reuse of software requirements. Objective: This study aims to identify and investigate the current state of the art with respect to (a) what requirement reuse approaches have been proposed, (b) the methods used to evaluate the approaches, (c) the characteristics of the approaches, and (d) the quality of empirical studies on requirements reuse with respect to rigor and relevance. Method: We conducted a systematic review and a combination of snowball sampling and database search have been used to identify the studies. The rigor and relevance scoring rubric has been used to assess the quality of the empirical studies. Multiple researchers have been involved in each step to increase the reliability of the study. Results: Sixty-nine studies were identified that describe requirements reuse approaches. The majority of the approaches used structuring and matching of requirements as a method to support requirements reuse and text-based artefacts were commonly used as an input to these approaches. Further evaluation of the studies revealed that the majority of the approaches are not validated in the industry. The subset of empirical studies (22 in total) was analyzed for rigor and relevance and two studies achieved the maximum score for rigor and relevance based on the rubric. It was found that mostly text-based requirements reuse approaches were validated in the industry. Conclusion: From the review, it was found that a number of approaches already exist in literature, but many approaches are not validated in industry. The evaluation of rigor and relevance of empirical studies show that these do not contain details of context, validity threats, and the industrial settings, thus highlighting the need for the industrial evaluation of the approaches. © 2017 Elsevier B.V.

Place, publisher, year, edition, pages
Elsevier, 2018
Keywords
Artefact reuse, Relevance, Requirements reuse, Reusability, Rigor, Software requirements, Quality control, Requirements engineering, Search engines, Computer software reusability
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-15356 (URN)10.1016/j.infsof.2017.09.009 (DOI)000414878200014 ()2-s2.0-85030849171 (Scopus ID)
Funder
Knowledge Foundation
Note

project "Professional Licentiate of Engineering Research School"

Available from: 2017-10-20 Created: 2017-10-20 Last updated: 2018-01-13Bibliographically approved
Marculescu, B., Feldt, R., Torkar, R. & Poulding, S. (2018). Transferring Interactive Search-Based Software Testing to Industry. Journal of Systems and Software, 142, 156-170
Open this publication in new window or tab >>Transferring Interactive Search-Based Software Testing to Industry
2018 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 142, p. 156-170Article in journal (Refereed) Published
Abstract [en]

Context: Search-Based Software Testing (SBST), and the wider area of Search-Based Software Engineering (SBSE), is the application of optimization algorithms to problems in software testing, and software engineering, respectively. New algorithms, methods, and tools are being developed and validated on benchmark problems. In previous work, we have also implemented and evaluated Interactive Search-Based Software Testing (ISBST) tool prototypes, with a goal to successfully transfer the technique to industry. Objective: While SBST and SBSE solutions are often validated on benchmark problems, there is a need to validate them in an operational setting, and to assess their performance in practice. The present paper discusses the development and deployment of SBST tools for use in industry, and reflects on the transfer of these techniques to industry. Method: In addition to previous work discussing the development and validation of an ISBST prototype, a new version of the prototype ISBST system was evaluated in the laboratory and in industry. This evaluation is based on an industrial System under Test (SUT) and was carried out with industrial practitioners. The Technology Transfer Model is used as a framework to describe the progression of the development and evaluation of the ISBST system, as it progresses through the first five of its seven steps. Results: The paper presents a synthesis of previous work developing and evaluating the ISBST prototype, as well as presenting an evaluation, in both academia and industry, of that prototype's latest version. In addition to the evaluation, the paper also discusses the lessons learned from this transfer. Conclusions: This paper presents an overview of the development and deployment of the ISBST system in an industrial setting, using the framework of the Technology Transfer Model. We conclude that the ISBST system is capable of evolving useful test cases for that setting, though improvements in the means the system uses to communicate that information to the user are still required. In addition, a set of lessons learned from the project are listed and discussed. Our objective is to help other researchers that wish to validate search-based systems in industry, and provide more information about the benefits and drawbacks of these systems.

Place, publisher, year, edition, pages
Elsevier, 2018
Keywords
interactive search-based software testing, industrial evaluation
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-15082 (URN)10.1016/j.jss.2018.04.061 (DOI)000437037200009 ()
Available from: 2017-08-30 Created: 2017-08-30 Last updated: 2018-08-20Bibliographically approved
Poulding, S. & Feldt, R. (2017). Automated Random Testing in Multiple Dispatch Languages. In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017: . Paper presented at 10th IEEE International Conference on Software Testing, Verification and Validation, ICST,Tokyo (pp. 333-344). IEEE Computer Society
Open this publication in new window or tab >>Automated Random Testing in Multiple Dispatch Languages
2017 (English)In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, IEEE Computer Society, 2017, p. 333-344Conference paper, Published paper (Refereed)
Abstract [en]

In programming languages that use multiple dispatch, a single function can have multiple implementations, each of which may specialise the function's operation. Which one of these implementations to execute is determined by the data types of all the arguments to the function. Effective testing of functions that use multiple dispatch therefore requires diverse test inputs in terms of the data types of the input's arguments as well as their values. In this paper we describe an approach for generating test inputs where both the values and types are chosen probabilistically. The approach uses reflection to automatically determine how to create inputs with the desired types, and dynamically updates the probability distribution from which types are sampled in order to improve both the test efficiency and efficacy. We evaluate the technique on 247 methods across 9 built-in functions of Julia, a technical computing language that applies multiple dispatch at runtime. In the process, we identify three real faults in these widely-used functions. © 2017 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2017
Keywords
Probability distributions, Verification, Built-in functions, Data type, Effective testing, Random testing, Runtimes, Technical computing, Test efficiency, Test inputs, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-14899 (URN)10.1109/ICST.2017.37 (DOI)000403393600030 ()2-s2.0-85020711461 (Scopus ID)9781509060313 (ISBN)
Conference
10th IEEE International Conference on Software Testing, Verification and Validation, ICST,Tokyo
Available from: 2017-07-06 Created: 2017-07-06 Last updated: 2018-01-13Bibliographically approved
Feldt, R., Poulding, S. M., Clark, D. & Yoo, S. (2016). Test Set Diameter: Quantifying the Diversity of Sets of Test Cases. In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST: . Paper presented at 9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago (pp. 223-233). IEEE Computer Society
Open this publication in new window or tab >>Test Set Diameter: Quantifying the Diversity of Sets of Test Cases
2016 (English)In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST, IEEE Computer Society, 2016, p. 223-233Conference paper, Published paper (Refereed)
Abstract [en]

A common and natural intuition among software testers is that test cases need to differ if a software system is to be tested properly and its quality ensured. Consequently, much research has gone into formulating distance measures for how test cases, their inputs and/or their outputs differ. However, common to these proposals is that they are data type specific and/or calculate the diversity only between pairs of test inputs, traces or outputs. We propose a new metric to measure the diversity of sets of tests: the test set diameter (TSDm). It extends our earlier, pairwise test diversity metrics based on recent advances in information theory regarding the calculation of the normalized compression distance (NCD) for multisets. A key advantage is that TSDm is a universal measure of diversity and so can be applied to any test set regardless of data type of the test inputs (and, moreover, to other test-related data such as execution traces). But this universality comes at the cost of greater computational effort compared to competing approaches. Our experiments on four different systems show that the test set diameter can help select test sets with higher structural and fault coverage than random selection even when only applied to test inputs. This can enable early test design and selection, prior to even having a software system to test, and complement other types of test automation and analysis. We argue that this quantification of test set diversity creates a number of opportunities to better understand software quality and provides practical ways to increase it.

Place, publisher, year, edition, pages
IEEE Computer Society, 2016
Series
IEEE International Conference on Software Testing Verification and Validation, ISSN 2381-2834
Keywords
Empirical study; Information theory; Software testing; Test selection
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-11201 (URN)10.1109/ICST.2016.33 (DOI)000391252900021 ()9781509018260 (ISBN)
Conference
9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago
Available from: 2016-01-14 Created: 2015-12-14 Last updated: 2018-01-10Bibliographically approved
Marculescu, B., Poulding, S., Feldt, R., Petersen, K. & Torkar, R. (2016). Tester interactivity makes a difference in search-based software testing: A controlled experiment. Information and Software Technology, 78, 66-82
Open this publication in new window or tab >>Tester interactivity makes a difference in search-based software testing: A controlled experiment
Show others...
2016 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 78, p. 66-82Article in journal (Refereed) Published
Abstract [en]

Context: Search-based software testing promises to provide users with the ability to generate high quality test cases, and hence increase product quality, with a minimal increase in the time and effort required. The development of the Interactive Search-Based Software Testing (ISBST) system was motivated by a previous study to investigate the application of search-based software testing (SBST) in an industrial setting. ISBST allows users to interact with the underlying SBST system, guiding the search and assessing the results. An industrial evaluation indicated that the ISBST system could find test cases that are not created by testers employing manual techniques. The validity of the evaluation was threatened, however, by the low number of participants. Objective: This paper presents a follow-up study, to provide a more rigorous evaluation of the ISBST system. Method: To assess the ISBST system a two-way crossover controlled experiment was conducted with 58 students taking a Verification and Validation course. The NASA Task Load Index (NASA-TLX) is used to assess the workload experienced by the participants in the experiment. Results:The experimental results validated the hypothesis that the ISBST system generates test cases that are not found by the same participants employing manual testing techniques. A follow-up laboratory experiment also investigates the importance of interaction in obtaining the results. In addition to this main result, the subjective workload was assessed for each participant by means of the NASA-TLX tool. The evaluation showed that, while the ISBST system required more effort from the participants, they achieved the same performance. Conclusions: The paper provides evidence that the ISBST system develops test cases that are not found by manual techniques, and that interaction plays an important role in achieving that result. (C) 2016 Elsevier B.V. All rights reserved.

Place, publisher, year, edition, pages
Elsevier, 2016
Keywords
Search-based software testing, Interactive search-based software testing, Controlled experiment
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-12958 (URN)10.1016/j.infsof.2016.05.009 (DOI)000379634100004 ()
External cooperation:
Available from: 2016-08-31 Created: 2016-08-30 Last updated: 2018-01-10Bibliographically approved
Marculescu, B., Feldt, R., Torkar, R. & Poulding, S. (2015). An initial industrial evaluation of interactive search-based testing for embedded software. Applied Soft Computing, 29, 26-39
Open this publication in new window or tab >>An initial industrial evaluation of interactive search-based testing for embedded software
2015 (English)In: Applied Soft Computing, ISSN 1568-4946, E-ISSN 1872-9681, Vol. 29, p. 26-39Article in journal (Refereed) Published
Abstract [en]

Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so. Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies. For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system. This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search. In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ’domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions. The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently. © 2014 Elsevier B.V. All rights reserved.

Keywords
Ability testing; Automatic test pattern generation; Embedded software; Personnel training; Search engines; Software engineering, Automated software testing; Education and training; Industrial experience; Interactive search; Knowledge and experience; Search-based algorithms; Search-based software testing; User-centered, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-718 (URN)10.1016/j.asoc.2014.12.025 (DOI)000350648100003 ()2-s2.0-84920672523 (Scopus ID)
Available from: 2015-06-01 Created: 2015-05-28 Last updated: 2018-01-11Bibliographically approved
Poulding, S. & Feldt, R. (2015). Heuristic Model Checking Using a Monte-Carlo Tree Search Algorithm. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation: . Paper presented at The Genetic and Evolutionary Computation Conference (GECCO), Madrid (pp. 1359-1366). ACM Digital Library
Open this publication in new window or tab >>Heuristic Model Checking Using a Monte-Carlo Tree Search Algorithm
2015 (English)In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, ACM Digital Library, 2015, p. 1359-1366Conference paper, Published paper (Refereed)
Place, publisher, year, edition, pages
ACM Digital Library, 2015
Keywords
model checking, nested monte-carlo search, search-based software engineering
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-11203 (URN)10.1145/2739480.2754767 (DOI)000358795700170 ()978-1-4503-3472-3 (ISBN)
Conference
The Genetic and Evolutionary Computation Conference (GECCO), Madrid
Available from: 2015-12-16 Created: 2015-12-14 Last updated: 2018-01-10Bibliographically approved
Clark, D., Feldt, R., Poulding, S. & Yoo, S. (2015). Information Transformation: An Underpinning Theory for Software Engineering. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol 2: . Paper presented at IEEE/ACM 2nd International Workshop on Software Architecture and Metrics (SAM), MAY 16-16, 2015, Florence, ITALY (pp. 599-602). IEEE
Open this publication in new window or tab >>Information Transformation: An Underpinning Theory for Software Engineering
2015 (English)In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol 2, IEEE , 2015, p. 599-602Conference paper, Published paper (Refereed)
Abstract [en]

Software engineering lacks underpinning scientific theories both for the software it produces and the processes by which it does so. We propose that an approach based on information theory can provide such a theory, or rather many theories. We envision that such a benefit will be realised primarily through research based on the quantification of information involved and a mathematical study of the limiting laws that arise. However, we also argue that less formal but more qualitative uses for information theory will be useful. The main argument in support of our vision is based on the fact that both a program and an engineering process to develop such a program are fundamentally processes that transform information. To illustrate our argument we focus on software testing and develop an initial theory in which a test suite is input/output adequate if it achieves the channel capacity of the program as measured by the mutual information between its inputs and its outputs. We outline a number of problems, metrics and concrete strategies for improving software engineering, based on information theoretical analyses. We find it likely that similar analyses and subsequent future research to detail them would be generally fruitful for software engineering.

Place, publisher, year, edition, pages
IEEE, 2015
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-14570 (URN)10.1109/ICSE.2015.202 (DOI)000380572400080 ()978-1-4799-1934-5 (ISBN)
Conference
IEEE/ACM 2nd International Workshop on Software Architecture and Metrics (SAM), MAY 16-16, 2015, Florence, ITALY
Available from: 2017-06-19 Created: 2017-06-19 Last updated: 2018-01-13Bibliographically approved
Poulding, S. & Feldt, R. (2015). Re-using generators of complex test data. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST): . Paper presented at 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015; Graz; Austria (pp. Article number 7102605). IEEE Computer Society
Open this publication in new window or tab >>Re-using generators of complex test data
2015 (English)In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), IEEE Computer Society, 2015, p. Article number 7102605-Conference paper, Published paper (Refereed)
Abstract [en]

The efficiency of random testing can be improved by sampling test inputs using a generating program that incorporates knowledge about the types of input most likely to detect faults in the software-under-test (SUT). But when the input of the SUT is a complex data type - such as a domain-specific string, array, record, tree, or graph - creating such a generator may be time- consuming and may require the tester to have substantial prior experience of the domain. In this paper we propose the re-use of generators created for one SUT on other SUTs that take the same complex data type as input. The re-use of a generator in this way would have little overhead, and we hypothesise that the re-used generator will typically be as least as efficient as the most straightforward form of random testing: sampling test inputs from the uniform distribution. We investigate this proposal for two data types using five generators. We assess test efficiency against seven real-world SUTs, and in terms of both structural coverage and the detection of seeded faults. The results support the re-use of generators for complex data types, and suggest that if a library of generators is to be maintained for this purpose, it is possible to extend library generators to accommodate the specific testing requirements of newly-encountered SUTs. © 2015 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2015
Series
IEEE International Conference on Software Testing Verification and Validation Workshops, ISSN 2159-4848
Keywords
Testing; Trees (mathematics); Verification, Domain specific; Library generators; Prior experience; Random testing; Sampling test; Test efficiency; Testing requirements; Uniform distribution, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-10674 (URN)10.1109/ICST.2015.7102605 (DOI)000380913300027 ()2-s2.0-84935074162 (Scopus ID)978-1-4799-7125-1 (ISBN)
Conference
8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015; Graz; Austria
Available from: 2015-09-16 Created: 2015-09-15 Last updated: 2018-01-11Bibliographically approved
Poulding, S., Alexander, R., Clark, J. A. & Hadley, M. J. (2015). The optimisation of stochastic grammars to enable cost-effective probabilistic structural testing. Journal of Systems and Software, 103, 296-310
Open this publication in new window or tab >>The optimisation of stochastic grammars to enable cost-effective probabilistic structural testing
2015 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 103, p. 296-310Article in journal (Refereed) Published
Abstract [en]

The effectiveness of statistical testing, a probabilistic structural testing strategy, depends on the characteristics of the probability distribution from which test inputs are sampled. Metaheuristic search has been shown to be a practical method of optimising the characteristics of such distributions. However, the applicability of the existing search-based algorithm is limited by the requirement that the software's inputs must be a fixed number of ordinal values. In this paper we propose a new algorithm that relaxes this limitation and so permits the derivation of probability distributions for a much wider range of software. The representation used by the new algorithm is based on a stochastic grammar supplemented with two novel features: conditional production weights and the dynamic partitioning of ordinal ranges. We demonstrate empirically that a search algorithm using this representation can optimise probability distributions over complex input domains and thereby enable costeffective statistical testing, and that the use of both conditional production weights and dynamic partitioning can be beneficial to the search process. (C) 2014 Elsevier Inc. All rights reserved.

Keywords
Search-based software engineering, Software testing, Grammar-based testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-698 (URN)10.1016/j.jss.2014.11.042 (DOI)000351971500020 ()
Available from: 2015-05-26 Created: 2015-05-26 Last updated: 2018-01-11Bibliographically approved
Organisations

Search in DiVA

Show all publications