Change search
Link to record
Permanent link

Direct link
BETA
Feldt, Robert
Alternative names
Publications (10 of 87) Show all publications
Marculescu, B. & Feldt, R. (2019). Finding a Boundary between Valid and Invalid Regions of the Input Space. In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC: . Paper presented at 25th Asia-Pacific Software Engineering Conference, APSEC 2018, Nara, Japan, 4 December 2018 through 7 December (pp. 169-178). IEEE Computer Society, Article ID 8719523.
Open this publication in new window or tab >>Finding a Boundary between Valid and Invalid Regions of the Input Space
2019 (English)In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC, IEEE Computer Society , 2019, p. 169-178, article id 8719523Conference paper, Published paper (Refereed)
Abstract [en]

In the context of robustness testing, the boundary between the valid and invalid regions of the input space can be an interesting source of erroneous inputs. Knowing where a specific software under test (SUT) has a boundary is also essential for validation in relation to requirements. However, finding where a SUT actually implements the boundary is a non-trivial problem that has not gotten much attention. This paper proposes a method of finding the boundary between the valid and invalid regions of the input space, by developing pairs of test sets that describe that boundary in detail. The proposed method consists of two steps. First, test data generators, directed by a search algorithm to maximise distance to known, valid test cases, generate valid test cases that are closer to the boundary. Second, these valid test cases undergo mutations to try to push them over the boundary and into the invalid part of the input space. This results in a pair of test sets, one consisting of test cases on the valid side of the boundary and a matched set on the outer side, with only a small distance between the two sets. The method is evaluated on a number of examples from the standard library of a modern programming language. We propose a method of determining the boundary between valid and invalid regions of the input space, and apply it on a SUT that has a non-contiguous valid region of the input space. From the small distance between the developed pairs of test sets, and the fact that one test set contains valid test cases and the other invalid test cases, we conclude that the pair of test sets described the boundary between the valid and invalid regions of that input space. Differences of behaviour can be observed between different distances and different sets of mutation operators, but all show that the method is able to identify the boundary between the valid and invalid regions of the input space. This is an important step towards more automated robustness testing. © 2018 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2019
Keywords
search based software testing, software testing, Testing, Input space, Mutation operators, Non trivial problems, Robustness testing, Search Algorithms, Search-based software testing, Standard libraries, Test sets
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-18159 (URN)10.1109/APSEC.2018.00031 (DOI)2-s2.0-85066806423 (Scopus ID)9781728119700 (ISBN)
Conference
25th Asia-Pacific Software Engineering Conference, APSEC 2018, Nara, Japan, 4 December 2018 through 7 December
Available from: 2019-06-20 Created: 2019-06-20 Last updated: 2019-06-20Bibliographically approved
Felderer, M., Marculescu, B., Gomes De Oliveira Neto, F., Feldt, R. & Torkar, R. (2018). A testability analysis framework for non-functional properties. In: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2018: . Paper presented at 11th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Västerås (pp. 54-58). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>A testability analysis framework for non-functional properties
Show others...
2018 (English)In: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 54-58Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2018
Keywords
Extra-Functional Properties, Non-Functional Properties, Robustness, Software Testing, Testability, Robustness (control systems), Verification, Non functional properties, Testability Analysis
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-16928 (URN)10.1109/ICSTW.2018.00028 (DOI)2-s2.0-85050964241 (Scopus ID)9781538663523 (ISBN)
Conference
11th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Västerås
Available from: 2018-08-21 Created: 2018-08-21 Last updated: 2018-08-21Bibliographically approved
Feldt, R., Zimmermann, T., Bergersen, G. R., Falessi, D., Jedlitschka, A., Juristo, N., . . . Turhan, B. (2018). Four commentaries on the use of students and professionals in empirical software engineering experiments. Journal of Empirical Software Engineering, 23(6), 3801-3820
Open this publication in new window or tab >>Four commentaries on the use of students and professionals in empirical software engineering experiments
Show others...
2018 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 23, no 6, p. 3801-3820Article in journal, Editorial material (Other academic) Published
Place, publisher, year, edition, pages
Springer New York LLC, 2018
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-17407 (URN)10.1007/s10664-018-9655-0 (DOI)000451593200020 ()2-s2.0-85057041427 (Scopus ID)
Available from: 2018-12-13 Created: 2018-12-13 Last updated: 2019-01-09Bibliographically approved
Marculescu, B., Feldt, R., Torkar, R. & Poulding, S. (2018). Transferring Interactive Search-Based Software Testing to Industry. Journal of Systems and Software, 142, 156-170
Open this publication in new window or tab >>Transferring Interactive Search-Based Software Testing to Industry
2018 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 142, p. 156-170Article in journal (Refereed) Published
Abstract [en]

Context: Search-Based Software Testing (SBST), and the wider area of Search-Based Software Engineering (SBSE), is the application of optimization algorithms to problems in software testing, and software engineering, respectively. New algorithms, methods, and tools are being developed and validated on benchmark problems. In previous work, we have also implemented and evaluated Interactive Search-Based Software Testing (ISBST) tool prototypes, with a goal to successfully transfer the technique to industry. Objective: While SBST and SBSE solutions are often validated on benchmark problems, there is a need to validate them in an operational setting, and to assess their performance in practice. The present paper discusses the development and deployment of SBST tools for use in industry, and reflects on the transfer of these techniques to industry. Method: In addition to previous work discussing the development and validation of an ISBST prototype, a new version of the prototype ISBST system was evaluated in the laboratory and in industry. This evaluation is based on an industrial System under Test (SUT) and was carried out with industrial practitioners. The Technology Transfer Model is used as a framework to describe the progression of the development and evaluation of the ISBST system, as it progresses through the first five of its seven steps. Results: The paper presents a synthesis of previous work developing and evaluating the ISBST prototype, as well as presenting an evaluation, in both academia and industry, of that prototype's latest version. In addition to the evaluation, the paper also discusses the lessons learned from this transfer. Conclusions: This paper presents an overview of the development and deployment of the ISBST system in an industrial setting, using the framework of the Technology Transfer Model. We conclude that the ISBST system is capable of evolving useful test cases for that setting, though improvements in the means the system uses to communicate that information to the user are still required. In addition, a set of lessons learned from the project are listed and discussed. Our objective is to help other researchers that wish to validate search-based systems in industry, and provide more information about the benefits and drawbacks of these systems.

Place, publisher, year, edition, pages
Elsevier, 2018
Keywords
interactive search-based software testing, industrial evaluation
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-15082 (URN)10.1016/j.jss.2018.04.061 (DOI)000437037200009 ()
Available from: 2017-08-30 Created: 2017-08-30 Last updated: 2018-08-20Bibliographically approved
Lenberg, P., Wallgren Tengberg, L. G. & Feldt, R. (2017). An initial analysis of software engineers’ attitudes towards organizational change. Journal of Empirical Software Engineering, 22(4), 2179-2205
Open this publication in new window or tab >>An initial analysis of software engineers’ attitudes towards organizational change
2017 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 22, no 4, p. 2179-2205Article in journal (Refereed) Published
Abstract [en]

Employees’ attitudes towards organizational change are a critical determinant in the change process. Researchers have therefore tried to determine what underlying concepts that affect them. These extensive efforts have resulted in the identification of several antecedents. However, no studies have been conducted in a software engineering context and the research has provided little information on the relative impact and importance of the identified concepts. In this study, we have combined results from previous social science research with results from software engineering research, and thereby identified three underlying concepts with an expected significant impact on software engineers’ attitudes towards organizational change, i.e. their knowledge about the intended change outcome, their understanding of the need for change, and their feelings of participation in the change process. The result of two separate multiple regression analysis, where we used industrial questionnaire data (N=56), showed that the attitude concept openness to change is predicted by all three concepts, while the attitude concept readiness for change is predicted by need for change and participation. Our research provides an empirical baseline to an important area of software engineering and the result can be a starting-point for future organizational change research. In addition, the proposed model prescribes practical directions for software engineering organizations to adopt in improving employees’ responses to change and, thus, increase the probability of a successful change.

Place, publisher, year, edition, pages
Springer, 2017
Keywords
Attitudes, Behavioral software engineering, Human aspects, Openness to change, Organizational change, Readiness for change, Social psychology, Software engineering, Systematic literature review, Regression analysis, Social sciences computing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13675 (URN)10.1007/s10664-016-9482-0 (DOI)000406290500016 ()2-s2.0-85006172824 (Scopus ID)
Available from: 2016-12-30 Created: 2016-12-30 Last updated: 2018-01-16Bibliographically approved
Poulding, S. & Feldt, R. (2017). Automated Random Testing in Multiple Dispatch Languages. In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017: . Paper presented at 10th IEEE International Conference on Software Testing, Verification and Validation, ICST,Tokyo (pp. 333-344). IEEE Computer Society
Open this publication in new window or tab >>Automated Random Testing in Multiple Dispatch Languages
2017 (English)In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, IEEE Computer Society, 2017, p. 333-344Conference paper, Published paper (Refereed)
Abstract [en]

In programming languages that use multiple dispatch, a single function can have multiple implementations, each of which may specialise the function's operation. Which one of these implementations to execute is determined by the data types of all the arguments to the function. Effective testing of functions that use multiple dispatch therefore requires diverse test inputs in terms of the data types of the input's arguments as well as their values. In this paper we describe an approach for generating test inputs where both the values and types are chosen probabilistically. The approach uses reflection to automatically determine how to create inputs with the desired types, and dynamically updates the probability distribution from which types are sampled in order to improve both the test efficiency and efficacy. We evaluate the technique on 247 methods across 9 built-in functions of Julia, a technical computing language that applies multiple dispatch at runtime. In the process, we identify three real faults in these widely-used functions. © 2017 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2017
Keywords
Probability distributions, Verification, Built-in functions, Data type, Effective testing, Random testing, Runtimes, Technical computing, Test efficiency, Test inputs, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-14899 (URN)10.1109/ICST.2017.37 (DOI)000403393600030 ()2-s2.0-85020711461 (Scopus ID)9781509060313 (ISBN)
Conference
10th IEEE International Conference on Software Testing, Verification and Validation, ICST,Tokyo
Available from: 2017-07-06 Created: 2017-07-06 Last updated: 2018-01-13Bibliographically approved
Gren, L., Torkar, R. & Feldt, R. (2017). Group development and group maturity when building agile teams: A qualitative and quantitative investigation at eight large companies. Journal of Systems and Software, 124, 104-119
Open this publication in new window or tab >>Group development and group maturity when building agile teams: A qualitative and quantitative investigation at eight large companies
2017 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 124, p. 104-119Article in journal (Refereed) Published
Abstract [en]

The agile approach to projects focuses more on close-knit teams than traditional waterfall projects, which means that aspects of group maturity become even more important. This psychological aspect is not much researched in connection to the building of an “agile team.” The purpose of this study is to investigate how building agile teams is connected to a group development model taken from social psychology. We conducted ten semi-structured interviews with coaches, Scrum Masters, and managers responsible for the agile process from seven different companies, and collected survey data from 66 group-members from four companies (a total of eight different companies). The survey included an agile measurement tool and the one part of the Group Development Questionnaire. The results show that the practitioners define group developmental aspects as key factors to a successful agile transition. Also, the quantitative measurement of agility was significantly correlated to the group maturity measurement. We conclude that adding these psychological aspects to the description of the “agile team” could increase the understanding of agility and partly help define an “agile team.” We propose that future work should develop specific guidelines for how software development teams at different maturity levels might adopt agile principles and practices differently.

Place, publisher, year, edition, pages
Elsevier, 2017
Keywords
Agile processes, Empirical study, Group psychology, Maturity, Measurement, Measurements, Software design, Surveys, Agile process, Empirical studies, Quantitative investigation, Quantitative measurement, Semi structured interviews, Software development teams, Software engineering
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13547 (URN)10.1016/j.jss.2016.11.024 (DOI)000390827200006 ()2-s2.0-84996563490 (Scopus ID)
Available from: 2016-12-07 Created: 2016-12-07 Last updated: 2018-01-13Bibliographically approved
Alégroth, E. & Feldt, R. (2017). On the long-term use of visual gui testing in industrial practice: a case study. Journal of Empirical Software Engineering, 22(6), 2937-2971
Open this publication in new window or tab >>On the long-term use of visual gui testing in industrial practice: a case study
2017 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 22, no 6, p. 2937-2971Article in journal (Refereed) Published
Abstract [en]

Visual GUI Testing (VGT) is a tool-driven technique for automated GUI-based testing that uses image recognition to interact with and assert the correctness of the behavior of a system through its GUI as it is shown to the user. The technique’s applicability, e.g. defect-finding ability, and feasibility, e.g. time to positive return on investment, have been shown through empirical studies in industrial practice. However, there is a lack of studies that evaluate the usefulness and challenges associated with VGT when used long-term (years) in industrial practice. This paper evaluates how VGT was adopted, applied and why it was abandoned at the music streaming application development company, Spotify, after several years of use. A qualitative study with two workshops and five well chosen employees is performed at the company, supported by a survey, which is analyzed with a grounded theory approach to answer the study’s three research questions. The interviews provide insights into the challenges, problems and limitations, but also benefits, that Spotify experienced during the adoption and use of VGT. However, due to the technique’s drawbacks, VGT has been abandoned for a new technique/framework, simply called the Test interface. The Test interface is considered more robust and flexible for Spotify’s needs but has several drawbacks, including that it does not test the actual GUI as shown to the user like VGT does. From the study’s results it is concluded that VGT can be used long-term in industrial practice but it requires organizational change as well as engineering best practices to be beneficial. Through synthesis of the study’s results, and results from previous work, a set of guidelines are presented that aim to aid practitioners to adopt and use VGT in industrial practice. However, due to the abandonment of the technique, future research is required to analyze in what types of projects the technique is, and is not, long-term viable. To this end, we also present Spotify’s Test interface solution for automated GUI-based testing and conclude that it has its own benefits and drawbacks.

Place, publisher, year, edition, pages
Springer, 2017
Keywords
Automated testing, Case study, Guidelines, Industrial, Visual GUI testing, Automation, Image recognition, Industrial management, Engineering best practice, Grounded theory approach, Industrial practices, Organizational change, Graphical user interfaces
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13894 (URN)10.1007/s10664-016-9497-6 (DOI)000411865400006 ()2-s2.0-85010807231 (Scopus ID)
Available from: 2017-02-22 Created: 2017-02-22 Last updated: 2018-01-13Bibliographically approved
Alégroth, E., Gustafsson, J., Ivarsson, H. & Feldt, R. (2017). Replicating Rare Software Failures with Exploratory Visual GUI Testing. IEEE Software, 34(5), 53-59, Article ID 8048660.
Open this publication in new window or tab >>Replicating Rare Software Failures with Exploratory Visual GUI Testing
2017 (English)In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 34, no 5, p. 53-59, article id 8048660Article in journal (Refereed) Published
Abstract [en]

Saab AB developed software that had a defect that manifested itself only after months of continuous system use. After years of customer failure reports, the defect still persisted, until Saab developed failure replication based on visual GUI testing. © 1984-2012 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2017
Keywords
failure replication, industrial success story, industry-academia collaboration, Saab AB, semiautomated testing, software development, software engineering, software testing, visual GUI testing, Defects, Graphical user interfaces, Software design, Continuous system, Failure reports, Software failure
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-15358 (URN)10.1109/MS.2017.3571568 (DOI)000411846300011 ()2-s2.0-85030544803 (Scopus ID)
Available from: 2017-10-20 Created: 2017-10-20 Last updated: 2018-01-13Bibliographically approved
Kashfi, P., Feldt, R., Nilsson, A. & Berntsson Svensson, R. (2016). A conceptual ux-aware model of requirements. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): . Paper presented at IFIP WG 13.2/13.5 Joint 6th International Conference on Human-Centered Software Engineering, HCSE 2016 and 8th International Conference on Human Error, Safety, and System Development, HESSD 2016; Stockholm; Sweden (pp. 234-245). Springer, 9856 LNCS
Open this publication in new window or tab >>A conceptual ux-aware model of requirements
2016 (English)In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer, 2016, Vol. 9856 LNCS, p. 234-245Conference paper, Published paper (Refereed)
Abstract [en]

User eXperience (UX) is becoming increasingly important for success of software products. Yet, many companies still face various challenges in their work with UX. Part of these challenges relate to inadequate knowledge and awareness of UX and that current UX models are commonly not practical nor well integrated into existing Software Engineering (SE) models and concepts. Therefore, we present a conceptual UX-aware model of requirements for software development practitioners. This layered model shows the interrelation between UX and functional and quality requirements. The model is developed based on current models of UX and software quality characteristics. Through the model we highlight the main differences between various requirement types in particular essentially subjective and accidentally subjective quality requirements. We also present the result of an initial validation of the model through interviews with 12 practitioners and researchers. Our results show that the model can raise practitioners’ knowledge and awareness of UX in particular in relation to requirement and testing activities. It can also facilitate UX-related communication among stakeholders with different backgrounds. © IFIP International Federation for Information Processing 2016.

Place, publisher, year, edition, pages
Springer, 2016
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN 0302-9743 ; 9856
Keywords
Computer software selection and evaluation; Errors; Safety engineering; Software engineering, Hedonic; Non instrumentals; Non-task-related; Quality requirements; Software Quality; Usability; User experience, Software design
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13204 (URN)10.1007/978-3-319-44902-9_15 (DOI)000389062800015 ()2-s2.0-84986208138 (Scopus ID)9783319449012 (ISBN)
Conference
IFIP WG 13.2/13.5 Joint 6th International Conference on Human-Centered Software Engineering, HCSE 2016 and 8th International Conference on Human Error, Safety, and System Development, HESSD 2016; Stockholm; Sweden
Available from: 2016-10-03 Created: 2016-10-03 Last updated: 2018-01-14Bibliographically approved
Organisations

Search in DiVA

Show all publications