Change search
Link to record
Permanent link

Direct link
BETA
Marculescu, Bogdan
Publications (10 of 13) Show all publications
Felderer, M., Marculescu, B., Gomes De Oliveira Neto, F., Feldt, R. & Torkar, R. (2018). A testability analysis framework for non-functional properties. In: 2018 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW): . Paper presented at 11th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Västerås (pp. 54-58). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>A testability analysis framework for non-functional properties
Show others...
2018 (English)In: 2018 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW), Institute of Electrical and Electronics Engineers Inc. , 2018, p. 54-58Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
Extra-Functional Properties, Non-Functional Properties, Robustness, Software Testing, Testability, Robustness (control systems), Verification, Non functional properties, Testability Analysis
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-16928 (URN)10.1109/ICSTW.2018.00028 (DOI)000492760300008 ()2-s2.0-85050964241 (Scopus ID)9781538663523 (ISBN)
Conference
11th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Västerås
Funder
Knowledge Foundation, 20130085
Available from: 2018-08-21 Created: 2018-08-21 Last updated: 2019-11-14Bibliographically approved
Marculescu, B. & Feldt, R. (2018). Finding a Boundary between Valid and Invalid Regions of the Input Space. In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC: . Paper presented at 25th Asia-Pacific Software Engineering Conference, APSEC 2018, Nara, Japan, 4 December 2018 through 7 December (pp. 169-178). IEEE Computer Society, Article ID 8719523.
Open this publication in new window or tab >>Finding a Boundary between Valid and Invalid Regions of the Input Space
2018 (English)In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC, IEEE Computer Society , 2018, p. 169-178, article id 8719523Conference paper, Published paper (Refereed)
Abstract [en]

In the context of robustness testing, the boundary between the valid and invalid regions of the input space can be an interesting source of erroneous inputs. Knowing where a specific software under test (SUT) has a boundary is also essential for validation in relation to requirements. However, finding where a SUT actually implements the boundary is a non-trivial problem that has not gotten much attention. This paper proposes a method of finding the boundary between the valid and invalid regions of the input space, by developing pairs of test sets that describe that boundary in detail. The proposed method consists of two steps. First, test data generators, directed by a search algorithm to maximise distance to known, valid test cases, generate valid test cases that are closer to the boundary. Second, these valid test cases undergo mutations to try to push them over the boundary and into the invalid part of the input space. This results in a pair of test sets, one consisting of test cases on the valid side of the boundary and a matched set on the outer side, with only a small distance between the two sets. The method is evaluated on a number of examples from the standard library of a modern programming language. We propose a method of determining the boundary between valid and invalid regions of the input space, and apply it on a SUT that has a non-contiguous valid region of the input space. From the small distance between the developed pairs of test sets, and the fact that one test set contains valid test cases and the other invalid test cases, we conclude that the pair of test sets described the boundary between the valid and invalid regions of that input space. Differences of behaviour can be observed between different distances and different sets of mutation operators, but all show that the method is able to identify the boundary between the valid and invalid regions of the input space. This is an important step towards more automated robustness testing. © 2018 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2018
Keywords
search based software testing, software testing, Testing, Input space, Mutation operators, Non trivial problems, Robustness testing, Search Algorithms, Search-based software testing, Standard libraries, Test sets
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-18159 (URN)10.1109/APSEC.2018.00031 (DOI)000474770300018 ()2-s2.0-85066806423 (Scopus ID)9781728119700 (ISBN)
Conference
25th Asia-Pacific Software Engineering Conference, APSEC 2018, Nara, Japan, 4 December 2018 through 7 December
Available from: 2019-06-20 Created: 2019-06-20 Last updated: 2019-09-10Bibliographically approved
Marculescu, B., Feldt, R., Torkar, R. & Poulding, S. (2018). Transferring Interactive Search-Based Software Testing to Industry. Journal of Systems and Software, 142, 156-170
Open this publication in new window or tab >>Transferring Interactive Search-Based Software Testing to Industry
2018 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 142, p. 156-170Article in journal (Refereed) Published
Abstract [en]

Context: Search-Based Software Testing (SBST), and the wider area of Search-Based Software Engineering (SBSE), is the application of optimization algorithms to problems in software testing, and software engineering, respectively. New algorithms, methods, and tools are being developed and validated on benchmark problems. In previous work, we have also implemented and evaluated Interactive Search-Based Software Testing (ISBST) tool prototypes, with a goal to successfully transfer the technique to industry. Objective: While SBST and SBSE solutions are often validated on benchmark problems, there is a need to validate them in an operational setting, and to assess their performance in practice. The present paper discusses the development and deployment of SBST tools for use in industry, and reflects on the transfer of these techniques to industry. Method: In addition to previous work discussing the development and validation of an ISBST prototype, a new version of the prototype ISBST system was evaluated in the laboratory and in industry. This evaluation is based on an industrial System under Test (SUT) and was carried out with industrial practitioners. The Technology Transfer Model is used as a framework to describe the progression of the development and evaluation of the ISBST system, as it progresses through the first five of its seven steps. Results: The paper presents a synthesis of previous work developing and evaluating the ISBST prototype, as well as presenting an evaluation, in both academia and industry, of that prototype's latest version. In addition to the evaluation, the paper also discusses the lessons learned from this transfer. Conclusions: This paper presents an overview of the development and deployment of the ISBST system in an industrial setting, using the framework of the Technology Transfer Model. We conclude that the ISBST system is capable of evolving useful test cases for that setting, though improvements in the means the system uses to communicate that information to the user are still required. In addition, a set of lessons learned from the project are listed and discussed. Our objective is to help other researchers that wish to validate search-based systems in industry, and provide more information about the benefits and drawbacks of these systems.

Place, publisher, year, edition, pages
Elsevier, 2018
Keywords
interactive search-based software testing, industrial evaluation
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-15082 (URN)10.1016/j.jss.2018.04.061 (DOI)000437037200009 ()
Available from: 2017-08-30 Created: 2017-08-30 Last updated: 2018-08-20Bibliographically approved
Marculescu, B., Jabbari, R. & Molléri, J. S. (2016). Perception of Scientific Evidence: Do Industry and Academia Share an Understanding?. In: : . Paper presented at Lärarlärdom 2016, Kristianstad.
Open this publication in new window or tab >>Perception of Scientific Evidence: Do Industry and Academia Share an Understanding?
2016 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Context: Collaboration depends on communication and upon having a similar understanding of the notions that are being discussed, and a similar appraisal of their value. Existing work seems to show that the collaboration between industry and academia is hampered by a difference in values. In particular, academic work focuses more on generalizing on the basis of existing evidence, while industry prefers to particularize conclusions to individual cases. This has lead to the conclusion that industry values scientific evidence less than academia does. 

Objective: This paper seeks to re-evaluate that conclusion, and investigate if industry and academia share a definition of scientific evidence. If evidence can be found of competing views, we propose a more finely grained model of empirical evidence and its role in building software engineering knowledge. Moreover, we seek to determine if a more nuanced look the notion of scientific evidence has an influence on how academics and industry practitioners perceive that notion. 

Method: We have developed a model of key concepts related to understanding empirical evidence in software engineering. An initial validation has been conducted, consisting of a survey of master students, to determine if competing views of evidence exist at that level. The model will be validated by further literature study and semistructured interviews with industry practitioners. 

Results: We propose a model of empirical evidence in software engineering, and initial validation of that model by means of a survey. The results of the survey indicate that conflicting opinions already exist in the student body regarding the notion of evidence, how trustworthy different sources of evidence and knowledge are, and which sources of evidence and types of evidence are more appropriate in various situations.

Conclusion: Rather than a difference in how industry and academia value scientific evidence, we see evidence of misunderstanding, of different notions of what constitutes scientific evidence and what strength of evidence is required to achieve specific goals. We propose a model of empirical evidence, to provide a better understanding of what is required in various situations and a better platform for communication between industry and academia.

National Category
Software Engineering
Identifiers
urn:nbn:se:bth-17647 (URN)
Conference
Lärarlärdom 2016, Kristianstad
Available from: 2019-02-27 Created: 2019-02-27 Last updated: 2019-03-08Bibliographically approved
Marculescu, B., Poulding, S., Feldt, R., Petersen, K. & Torkar, R. (2016). Tester interactivity makes a difference in search-based software testing: A controlled experiment. Information and Software Technology, 78, 66-82
Open this publication in new window or tab >>Tester interactivity makes a difference in search-based software testing: A controlled experiment
Show others...
2016 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 78, p. 66-82Article in journal (Refereed) Published
Abstract [en]

Context: Search-based software testing promises to provide users with the ability to generate high quality test cases, and hence increase product quality, with a minimal increase in the time and effort required. The development of the Interactive Search-Based Software Testing (ISBST) system was motivated by a previous study to investigate the application of search-based software testing (SBST) in an industrial setting. ISBST allows users to interact with the underlying SBST system, guiding the search and assessing the results. An industrial evaluation indicated that the ISBST system could find test cases that are not created by testers employing manual techniques. The validity of the evaluation was threatened, however, by the low number of participants. Objective: This paper presents a follow-up study, to provide a more rigorous evaluation of the ISBST system. Method: To assess the ISBST system a two-way crossover controlled experiment was conducted with 58 students taking a Verification and Validation course. The NASA Task Load Index (NASA-TLX) is used to assess the workload experienced by the participants in the experiment. Results:The experimental results validated the hypothesis that the ISBST system generates test cases that are not found by the same participants employing manual testing techniques. A follow-up laboratory experiment also investigates the importance of interaction in obtaining the results. In addition to this main result, the subjective workload was assessed for each participant by means of the NASA-TLX tool. The evaluation showed that, while the ISBST system required more effort from the participants, they achieved the same performance. Conclusions: The paper provides evidence that the ISBST system develops test cases that are not found by manual techniques, and that interaction plays an important role in achieving that result. (C) 2016 Elsevier B.V. All rights reserved.

Place, publisher, year, edition, pages
Elsevier, 2016
Keywords
Search-based software testing, Interactive search-based software testing, Controlled experiment
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-12958 (URN)10.1016/j.infsof.2016.05.009 (DOI)000379634100004 ()
External cooperation:
Available from: 2016-08-31 Created: 2016-08-30 Last updated: 2018-01-10Bibliographically approved
Marculescu, B., Feldt, R. & Torkar, R. (2016). Using Exploration Focused Techniques to Augment Search-Based Software Testing: An Experimental Evaluation. In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016: . Paper presented at 9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago (pp. 69-79). IEEE Computer Society, Article ID 7515460.
Open this publication in new window or tab >>Using Exploration Focused Techniques to Augment Search-Based Software Testing: An Experimental Evaluation
2016 (English)In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016, IEEE Computer Society, 2016, p. 69-79, article id 7515460Conference paper, Published paper (Refereed)
Abstract [en]

Search-based software testing (SBST) often uses objective-based approaches to solve testing problems. There are, however, situations where the validity and completeness of objectives cannot be ascertained, or where there is insufficient information to define objectives at all. Incomplete or incorrect objectives may steer the search away from interesting behavior of the software under test (SUT) and from potentially useful test cases. This papers investigates the degree to which exploration-based algorithms can be used to complement an objective-based tool we have previously developed and evaluated in industry. In particular, we would like to assess how exploration-based algorithms perform in situations where little information on the behavior space is available a priori. We have conducted an experiment comparing the performance of an exploration-based algorithm with an objective-based one on a problem with a high-dimensional behavior space. In addition, we evaluate to what extent that performance degrades in situations where computational resources are limited. Our experiment shows that exploration-based algorithms are useful in covering a larger area of the behavior space and result in a more diverse solution population. Typically, of the candidate solutions that exploration-based algorithms propose, more than 80% were not covered by their objective-based counterpart. This increased diversity is present in the resulting population even when computational resources are limited. We conclude that exploration-focused algorithms are a useful means of investigating high-dimensional spaces, even in situations where limited information and limited resources are available.

Place, publisher, year, edition, pages
IEEE Computer Society, 2016
Series
IEEE International Conference on Software Testing Verification and Validation, ISSN 2381-2834
Keywords
Algorithms; Verification, Computational resources; Controlled experiment; Diverse solutions; Experimental evaluation; High dimensional spaces; High-dimensional; Limited information; Search-based software testing, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13094 (URN)10.1109/ICST.2016.26 (DOI)000391252900007 ()2-s2.0-84983246859 (Scopus ID)9780306406157 (ISBN)
Conference
9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago
Available from: 2016-10-06 Created: 2016-10-03 Last updated: 2018-01-14Bibliographically approved
Marculescu, B., Feldt, R., Torkar, R. & Poulding, S. (2015). An initial industrial evaluation of interactive search-based testing for embedded software. Applied Soft Computing, 29, 26-39
Open this publication in new window or tab >>An initial industrial evaluation of interactive search-based testing for embedded software
2015 (English)In: Applied Soft Computing, ISSN 1568-4946, E-ISSN 1872-9681, Vol. 29, p. 26-39Article in journal (Refereed) Published
Abstract [en]

Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so. Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies. For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system. This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search. In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ’domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions. The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently. © 2014 Elsevier B.V. All rights reserved.

Keywords
Ability testing; Automatic test pattern generation; Embedded software; Personnel training; Search engines; Software engineering, Automated software testing; Education and training; Industrial experience; Interactive search; Knowledge and experience; Search-based algorithms; Search-based software testing; User-centered, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-718 (URN)10.1016/j.asoc.2014.12.025 (DOI)000350648100003 ()2-s2.0-84920672523 (Scopus ID)
Available from: 2015-06-01 Created: 2015-05-28 Last updated: 2018-01-11Bibliographically approved
Marculescu, B. (2014). Interactive Search-Based Testing of Embedded Software: Understanding the Benefits and Difficulties of Industrial Application. (Licentiate dissertation). Karlskrona: Blekinge Institute of Technology
Open this publication in new window or tab >>Interactive Search-Based Testing of Embedded Software: Understanding the Benefits and Difficulties of Industrial Application
2014 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

The ubiquity of software has wide-ranging consequences for its development and testing. Increasingly often, software is developed and tested by engineers specialized in other areas. Embedded software, for example, is developed ad-hoc, for each product, by systems and domain engineers. Supporting testing activities in this context requires a highly flexible approach, powerful enough to create useful test cases, yet simple enough to not require specialized training in software testing. Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. It is, however, a set of complex techniques that cannot be used off-the-shelf as part of the software development process of a company. The objective of the work presented in this thesis is to investigate the applicability of Search-Based Software Testing in an industrial environment. A second objective was identifying additional knowledge gaps relating to integrating such techniques in existing software development processes. Investigating how meaningful interaction is to take place, what information is needed in order for both stakeholders to be able to achieve their objectives is a third goal. The findings are obtained by means of a series of case studies in a company developing both embedded software and the tools to enable embedded software development. A prototype Interactive Search-Based Software Testing (ISBST) system was developed that uses user interaction to drive the search-based component towards interesting test cases. The ISBST system was evaluated constantly, and improved based on the findings of each case study. The latest case study was an empirical evaluation of the system with the engineers, both software engineers and domain specialists, in the company. The empirical work includes both qualitative and quantitative data, with a focus on the exploratory study of the practical factors affecting the use of the ISBST system. A key early finding is that interactivity is essential when implementing search-based techniques in the industrial context described above. Repeated validations conducted with the company yielded additional information on the practicalities of interaction. The strength of SBST proved useful in investigating areas of the test space that were normally overlooked due to limitations in terms of resources. At the same time, developers were able to use their experience and intuition to guide the SBST system towards test cases that were more likely to be problematic. Overall, the results obtained indicate that the search-based techniques provide a useful complement to existing testing techniques. SBST, in its interactive form, can be a useful complement to existing testing techniques. An Interactive SBST (ISBST) system has been developed as a result of this research. Results show that this system is usable by the developers of embedded software, that often specialize on acquiring domain knowledge rather than software engineering expertise.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Institute of Technology, 2014. p. 93 p.
Series
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 1
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-00579 (URN)oai:bth.se:forskinfoE44DEFE0686B8A70C1257C6A00467ABC (Local ID)978-91-7295-274-4 (ISBN)oai:bth.se:forskinfoE44DEFE0686B8A70C1257C6A00467ABC (Archive number)oai:bth.se:forskinfoE44DEFE0686B8A70C1257C6A00467ABC (OAI)
Available from: 2014-04-10 Created: 2014-01-24 Last updated: 2018-01-11Bibliographically approved
Marculescu, B., Feldt, R. & Torkar, R. (2013). Objective Re-Weighting to Guide an Interactive Search Based Software Testing System. Paper presented at 12th International Conference on Machine Learning and Applications (ICMLA). Paper presented at 12th International Conference on Machine Learning and Applications (ICMLA). Miami: IEEE
Open this publication in new window or tab >>Objective Re-Weighting to Guide an Interactive Search Based Software Testing System
2013 (English)Conference paper, Published paper (Refereed) Published
Abstract [en]

Even hardware-focused industries today develop products where software is both a large and important component. Engineers tasked with developing and integrating these products do not always have a software engineering background. To ensure quality, tools are needed that automate and support software testing while allowing these domain specialists to leverage their knowledge and experience. Search-based testing could be a key aspect in creating an automated tool for supporting testing activities. However, domain specific quality criteria and trade-offs make it difficult to develop a general fitness function a priori, so interaction between domain specialists and such a tool would be critical to its success. In this paper we present a system for interactive search-based software testing and investigate a way for domain specialists to guide the search by dynamically re-weighting quality goals. Our empirical investigation shows that objective reweighting can help a human domain specialist interactively guide the search, without requiring specialized knowledge of the system and without sacrificing population diversity.

Place, publisher, year, edition, pages
Miami: IEEE, 2013
Keywords
search based software testing, interactive search based software engineering, user centered, embedded software, industrial experience
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-6354 (URN)10.1109/ICMLA.2013.113 (DOI)000353638700017 ()oai:bth.se:forskinfo3A5EEB62A60C7D4CC1257E50004D04D9 (Local ID)978-0-7695-5144-9 (ISBN)oai:bth.se:forskinfo3A5EEB62A60C7D4CC1257E50004D04D9 (Archive number)oai:bth.se:forskinfo3A5EEB62A60C7D4CC1257E50004D04D9 (OAI)
Conference
12th International Conference on Machine Learning and Applications (ICMLA)
Available from: 2015-05-25 Created: 2015-05-25 Last updated: 2018-01-11Bibliographically approved
Marculescu, B., Feldt, R. & Torkar, R. (2013). Practitioner-Oriented Visualization in an Interactive Search-Based Software Test Creation Tool. In: : . Paper presented at 20th Asia - Pacific Software Engineering Conference (APSEC), Bangkok (pp. 87-92). IEEE Computer Society, 2
Open this publication in new window or tab >>Practitioner-Oriented Visualization in an Interactive Search-Based Software Test Creation Tool
2013 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Search-based software testing uses meta-heuristic search techniques to automate or partially automate testing tasks, such as test case generation or test data generation. It uses a fitness function to encode the quality characteristics that are relevant, for a given problem, and guides the search to acceptable solutions in a potentially vast search space. From an industrial perspective, this opens up the possibility of generating and evaluating lots of test cases without raising costs to unacceptable levels. First, however, the applicability of search-based software engineering in an industrial setting must be evaluated. In practice, it is difficult to develop a priori a fitness function that covers all practical aspects of a problem. Interaction with human experts offers access to experience that is otherwise unavailable and allows the creation of a more informed and accurate fitness function. Moreover, our industrial partner has already expressed a view that the knowledge and experience of domain specialists are more important to the overall quality of the systems they develop than software engineering expertise. In this paper we describe our application of Interactive Search Based Software Testing (ISBST) in an industrial setting. We used SBST to search for test cases for an industrial software module and based, in part, on interaction with a human domain specialist. Our evaluation showed that such an approach is feasible, though it also identified potential difficulties relating to the interaction between the domain specialist and the system.

Place, publisher, year, edition, pages
IEEE Computer Society, 2013
Keywords
interactive systems, program testing, ISBST, industrial software module, interactive search-based software test creation tool, meta-heuristic search techniques, practitioner-oriented visualization
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-11366 (URN)10.1109/APSEC.2013.120 (DOI)000358735300018 ()
Conference
20th Asia - Pacific Software Engineering Conference (APSEC), Bangkok
Funder
Knowledge Foundation, 2010/0124
Available from: 2016-01-05 Created: 2016-01-05 Last updated: 2018-01-17Bibliographically approved
Organisations

Search in DiVA

Show all publications