Change search
Link to record
Permanent link

Direct link
BETA
Torkar, Richard
Alternative names
Publications (10 of 52) Show all publications
Gren, L., Torkar, R. & Feldt, R. (2017). Group development and group maturity when building agile teams: A qualitative and quantitative investigation at eight large companies. Journal of Systems and Software, 124, 104-119
Open this publication in new window or tab >>Group development and group maturity when building agile teams: A qualitative and quantitative investigation at eight large companies
2017 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 124, p. 104-119Article in journal (Refereed) Published
Abstract [en]

The agile approach to projects focuses more on close-knit teams than traditional waterfall projects, which means that aspects of group maturity become even more important. This psychological aspect is not much researched in connection to the building of an “agile team.” The purpose of this study is to investigate how building agile teams is connected to a group development model taken from social psychology. We conducted ten semi-structured interviews with coaches, Scrum Masters, and managers responsible for the agile process from seven different companies, and collected survey data from 66 group-members from four companies (a total of eight different companies). The survey included an agile measurement tool and the one part of the Group Development Questionnaire. The results show that the practitioners define group developmental aspects as key factors to a successful agile transition. Also, the quantitative measurement of agility was significantly correlated to the group maturity measurement. We conclude that adding these psychological aspects to the description of the “agile team” could increase the understanding of agility and partly help define an “agile team.” We propose that future work should develop specific guidelines for how software development teams at different maturity levels might adopt agile principles and practices differently.

Place, publisher, year, edition, pages
Elsevier, 2017
Keywords
Agile processes, Empirical study, Group psychology, Maturity, Measurement, Measurements, Software design, Surveys, Agile process, Empirical studies, Quantitative investigation, Quantitative measurement, Semi structured interviews, Software development teams, Software engineering
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13547 (URN)10.1016/j.jss.2016.11.024 (DOI)000390827200006 ()2-s2.0-84996563490 (Scopus ID)
Available from: 2016-12-07 Created: 2016-12-07 Last updated: 2018-01-13Bibliographically approved
Afzal, W. & Torkar, R. (2016). Towards benchmarking feature subset selection methods for software fault prediction (617ed.). In: Studies in Computational Intelligence: (pp. 33-58). Springer, 617
Open this publication in new window or tab >>Towards benchmarking feature subset selection methods for software fault prediction
2016 (English)In: Studies in Computational Intelligence, Springer, 2016, 617, Vol. 617, p. 33-58Chapter in book (Refereed)
Abstract [en]

Despite the general acceptance that software engineering datasets often contain noisy, irrelevant or redundant variables, very few benchmark studies of feature subset selection (FSS) methods on real-life data from software projects have been conducted. This paper provides an empirical comparison of state-of-the-art FSS methods: information gain attribute ranking (IG); Relief (RLF); principal component analysis (PCA); correlation-based feature selection (CFS); consistencybased subset evaluation (CNS); wrapper subset evaluation (WRP); and an evolutionary computation method, genetic programming (GP), on five fault prediction datasets from the PROMISE data repository. For all the datasets, the area under the receiver operating characteristic curve—the AUC value averaged over 10-fold cross-validation runs—was calculated for each FSS method-dataset combination before and after FSS. Two diverse learning algorithms, C4.5 and naïve Bayes (NB) are used to test the attribute sets given by each FSS method. The results show that although there are no statistically significant differences between the AUC values for the different FSS methods for both C4.5 and NB, a smaller set of FSS methods (IG, RLF, GP) consistently select fewer attributes without degrading classification accuracy. We conclude that in general, FSS is beneficial as it helps improve classification accuracy of NB and C4.5. There is no single best FSS method for all datasets but IG, RLF and GP consistently select fewer attributes without degrading classification accuracy within statistically significant boundaries. © Springer International Publishing Switzerland 2016.

Place, publisher, year, edition, pages
Springer, 2016 Edition: 617
Series
Studies in Computational Intelligence, ISSN 1860-949X ; 617
Keywords
Empirical; Fault prediction; Feature subset selection
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-11573 (URN)10.1007/978-3-319-25964-2_3 (DOI)2-s2.0-84955278082 (Scopus ID)
External cooperation:
Available from: 2016-02-03 Created: 2016-02-03 Last updated: 2018-01-10Bibliographically approved
Marculescu, B., Feldt, R. & Torkar, R. (2016). Using Exploration Focused Techniques to Augment Search-Based Software Testing: An Experimental Evaluation. In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016: . Paper presented at 9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago (pp. 69-79). IEEE Computer Society, Article ID 7515460.
Open this publication in new window or tab >>Using Exploration Focused Techniques to Augment Search-Based Software Testing: An Experimental Evaluation
2016 (English)In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016, IEEE Computer Society, 2016, p. 69-79, article id 7515460Conference paper, Published paper (Refereed)
Abstract [en]

Search-based software testing (SBST) often uses objective-based approaches to solve testing problems. There are, however, situations where the validity and completeness of objectives cannot be ascertained, or where there is insufficient information to define objectives at all. Incomplete or incorrect objectives may steer the search away from interesting behavior of the software under test (SUT) and from potentially useful test cases. This papers investigates the degree to which exploration-based algorithms can be used to complement an objective-based tool we have previously developed and evaluated in industry. In particular, we would like to assess how exploration-based algorithms perform in situations where little information on the behavior space is available a priori. We have conducted an experiment comparing the performance of an exploration-based algorithm with an objective-based one on a problem with a high-dimensional behavior space. In addition, we evaluate to what extent that performance degrades in situations where computational resources are limited. Our experiment shows that exploration-based algorithms are useful in covering a larger area of the behavior space and result in a more diverse solution population. Typically, of the candidate solutions that exploration-based algorithms propose, more than 80% were not covered by their objective-based counterpart. This increased diversity is present in the resulting population even when computational resources are limited. We conclude that exploration-focused algorithms are a useful means of investigating high-dimensional spaces, even in situations where limited information and limited resources are available.

Place, publisher, year, edition, pages
IEEE Computer Society, 2016
Series
IEEE International Conference on Software Testing Verification and Validation, ISSN 2381-2834
Keywords
Algorithms; Verification, Computational resources; Controlled experiment; Diverse solutions; Experimental evaluation; High dimensional spaces; High-dimensional; Limited information; Search-based software testing, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13094 (URN)10.1109/ICST.2016.26 (DOI)000391252900007 ()2-s2.0-84983246859 (Scopus ID)9780306406157 (ISBN)
Conference
9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago
Available from: 2016-10-06 Created: 2016-10-03 Last updated: 2018-01-14Bibliographically approved
Afzal, W., Ghazi, A. N., Itkonen, J., Torkar, R., Andrews, A. & Bhatti, K. (2015). An experiment on the effectiveness and efficiency of exploratory testing. Empirical Software Engineering, 20(3), 844-878
Open this publication in new window or tab >>An experiment on the effectiveness and efficiency of exploratory testing
Show others...
2015 (English)In: Empirical Software Engineering, ISSN 1382-3256, Vol. 20, no 3, p. 844-878Article in journal (Refereed) Published
Abstract [en]

The exploratory testing (ET) approach is commonly applied in industry, but lacks scientific research. The scientific community needs quantitative results on the performance of ET taken from realistic experimental settings. The objective of this paper is to quantify the effectiveness and efficiency of ET vs. testing with documented test cases (test case based testing, TCT). We performed four controlled experiments where a total of 24 practitioners and 46 students performed manual functional testing using ET and TCT. We measured the number of identified defects in the 90-minute testing sessions, the detection difficulty, severity and types of the detected defects, and the number of false defect reports. The results show that ET found a significantly greater number of defects. ET also found significantly more defects of varying levels of difficulty, types and severity levels. However, the two testing approaches did not differ significantly in terms of the number of false defect reports submitted. We conclude that ET was more efficient than TCT in our experiment. ET was also more effective than TCT when detection difficulty, type of defects and severity levels are considered. The two approaches are comparable when it comes to the number of false defect reports submitted.

Place, publisher, year, edition, pages
Springer, 2015
Keywords
Software testing, Experiment, Exploratory testing, Efficiency, Effectiveness
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-6727 (URN)10.1007/s10664-014-9301-4 (DOI)000354480800008 ()oai:bth.se:forskinfoAF25267D19C21440C1257CBF007311E9 (Local ID)oai:bth.se:forskinfoAF25267D19C21440C1257CBF007311E9 (Archive number)oai:bth.se:forskinfoAF25267D19C21440C1257CBF007311E9 (OAI)
Available from: 2014-04-22 Created: 2014-04-19 Last updated: 2018-01-11Bibliographically approved
Marculescu, B., Feldt, R., Torkar, R. & Poulding, S. (2015). An initial industrial evaluation of interactive search-based testing for embedded software. Applied Soft Computing, 29, 26-39
Open this publication in new window or tab >>An initial industrial evaluation of interactive search-based testing for embedded software
2015 (English)In: Applied Soft Computing, ISSN 1568-4946, E-ISSN 1872-9681, Vol. 29, p. 26-39Article in journal (Refereed) Published
Abstract [en]

Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so. Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies. For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system. This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search. In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ’domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions. The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently. © 2014 Elsevier B.V. All rights reserved.

Keywords
Ability testing; Automatic test pattern generation; Embedded software; Personnel training; Search engines; Software engineering, Automated software testing; Education and training; Industrial experience; Interactive search; Knowledge and experience; Search-based algorithms; Search-based software testing; User-centered, Software testing
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-718 (URN)10.1016/j.asoc.2014.12.025 (DOI)000350648100003 ()2-s2.0-84920672523 (Scopus ID)
Available from: 2015-06-01 Created: 2015-05-28 Last updated: 2018-01-11Bibliographically approved
Gren, L., Torkar, R. & Feldt, R. (2015). Group Maturity and Agility, Are They Connected?: – A Survey Study. In: Proceedings of the 41st EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA): . Paper presented at 41st Euromicro Conference on Software Engineering and Advanced Applications 2015 (SEAA),Funchal, Madeira, Portugal (pp. 1-8). IEEE
Open this publication in new window or tab >>Group Maturity and Agility, Are They Connected?: – A Survey Study
2015 (English)In: Proceedings of the 41st EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2015, p. 1-8Conference paper, Published paper (Refereed)
Abstract [en]

The focus on psychology has increased within software engineering due to the project management innovation "agile development processes". The agile methods do not explicitly consider group development aspects; they simply assume what is described in group psychology as mature groups. This study was conducted with 45 employees and their twelve managers (N=57) from two SAP customers in the US that were working with agile methods, and the data were collected via an online survey. The selected Agility measurement was correlated to a Group Development measurement and showed significant convergent validity, i.e., a more mature team is also a more agile team. This means that the agile methods probably would benefit from taking group development into account when its practices are being introduced.

Place, publisher, year, edition, pages
IEEE, 2015
Keywords
FACULTY GROUP DEVELOPMENT; ORGANIZATIONAL CULTURE; PRODUCTIVITY; TEAMWORK; LINK
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-11210 (URN)10.1109/SEAA.2015.31 (DOI)000380478300001 ()978-1-4673-7585-6 (ISBN)
Conference
41st Euromicro Conference on Software Engineering and Advanced Applications 2015 (SEAA),Funchal, Madeira, Portugal
Available from: 2015-12-15 Created: 2015-12-14 Last updated: 2018-01-10Bibliographically approved
Lucas, G., Torkar, R. & Robert, F. (2015). The prospects of a quantitative measurement of agility: A validation study on an agile maturity model. Journal of Systems and Software, 107, 38-49
Open this publication in new window or tab >>The prospects of a quantitative measurement of agility: A validation study on an agile maturity model
2015 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 107, p. 38-49Article in journal (Refereed) Published
Abstract [en]

Agile development has now become a well-known approach to collaboration in professional work life. Both researchers and practitioners want validated tools to measure agility. This study sets out to validate an agile maturity measurement model with statistical tests and empirical data. First, a pretest was conducted as a case study including a survey and focus group. Second, the main study was conducted with 45 employees from two SAP customers in the US. We used internal consistency (by a Cronbach’s alpha) as the main measure for reliability and analyzed construct validity by exploratory principal factor analysis (PFA). The results suggest a new categorization of a subset of items existing in the tool and provides empirical support for these new groups of factors. However, we argue that more work is needed to reach the point where a maturity models with quantitative data can be said to validly measure agility, and even then, such a measurement still needs to include some deeper analysis with cultural and contextual items.

Place, publisher, year, edition, pages
Elsevier, 2015
Keywords
Agility; Empirical study; Validation
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-769 (URN)10.1016/j.jss.2015.05.008 (DOI)000358699700003 ()
Available from: 2015-06-04 Created: 2015-06-04 Last updated: 2018-01-11Bibliographically approved
Holt, N. E., Briand, L. & Torkar, R. (2014). Empirical evaluations on the cost-effectiveness of state-based testing: An industrial case study. Information and Software Technology, 56(8), 890-910
Open this publication in new window or tab >>Empirical evaluations on the cost-effectiveness of state-based testing: An industrial case study
2014 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 56, no 8, p. 890-910Article in journal (Refereed) Published
Abstract [en]

Context Test models describe the expected behavior of the software under test and provide the basis for test case and oracle generation. When test models are expressed as UML state machines, this is typically referred to as state-based testing (SBT). Despite the importance of being systematic while testing, all testing activities are limited by resource constraints. Thus, reducing the cost of testing while ensuring sufficient fault detection is a common goal in software development. No rigorous industrial case studies of SBT have yet been published. Objective In this paper, we evaluate the cost-effectiveness of SBT on actual control software by studying the combined influence of four testing aspects: coverage criterion, test oracle, test model and unspecified behavior (sneak paths). Method An industrial case study was used to investigate the cost-effectiveness of SBT. To enable the evaluation of SBT techniques, a model-based testing tool was configured and used to automatically generate test suites. The test suites were evaluated using 26 real faults collected in a field study. Results Results show that the more detailed and rigorous the test model and oracle, the higher the fault-detection ability of SBT. A less precise oracle achieved 67% fault detection, but the overall cost reduction of 13% was not enough to make the loss an acceptable trade-off. Removing details from the test model significantly reduced the cost by 85%. Interestingly, only a 24–37% reduction in fault detection was observed. Testing for sneak paths killed the remaining eleven mutants that could not be killed by the conformance test strategies. Conclusions Each of the studied testing aspects influences cost-effectiveness and must be carefully considered in context when selecting strategies. Regardless of these choices, sneak-path testing is a necessary step in SBT since sneak paths are common while also undetectable by conformance testing.

Place, publisher, year, edition, pages
Elsevier, 2014
Keywords
State-based testing, UML, Cost effectiveness, Automated testing, Empirical evaluation, Industrial case study
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-6671 (URN)10.1016/j.infsof.2014.02.011 (DOI)000337261800005 ()oai:bth.se:forskinfo0EC9153D017C0DA7C1257CDD0049ECDA (Local ID)oai:bth.se:forskinfo0EC9153D017C0DA7C1257CDD0049ECDA (Archive number)oai:bth.se:forskinfo0EC9153D017C0DA7C1257CDD0049ECDA (OAI)
Available from: 2014-07-17 Created: 2014-05-19 Last updated: 2018-01-11Bibliographically approved
Madeyski, L., Orzeszyna, W., Torkar, R. & Józala, M. (2014). Overcoming the equivalent mutant problem: A systematic literature review and a comparative experiment of second order mutation. IEEE Transactions on Software Engineering, 40(1), 23-42
Open this publication in new window or tab >>Overcoming the equivalent mutant problem: A systematic literature review and a comparative experiment of second order mutation
2014 (English)In: IEEE Transactions on Software Engineering, ISSN 0098-5589, E-ISSN 1939-3520, Vol. 40, no 1, p. 23-42Article in journal (Refereed) Published
Abstract [en]

Context. The equivalent mutant problem (EMP) is one of the crucial problems in mutation testing widely studied over decades. Objectives. The objectives are: to present a systematic literature review (SLR) in the field of EMP; to identify, classify and improve the existing, or implement new, methods which try to overcome EMP and evaluate them. Method. We performed SLR based on the search of digital libraries. We implemented four second order mutation (SOM) strategies, in addition to first order mutation (FOM), and compared them from different perspectives. Results. Our SLR identified 17 relevant techniques (in 22 articles) and three categories of techniques: detecting (DEM); suggesting (SEM); and avoiding equivalent mutant generation (AEMG). The experiment indicated that SOM in general and JudyDiffOp strategy in particular provide the best results in the following areas: total number of mutants generated; the association between the type of mutation strategy and whether the generated mutants were equivalent or not; the number of not killed mutants; mutation testing time; time needed for manual classification. Conclusions. The results in the DEM category are still far from perfect. Thus, the SEM and AEMG categories have been developed. The JudyDiffOp algorithm achieved good results in many areas.

Place, publisher, year, edition, pages
IEEE, 2014
Keywords
mutation testing, equivalent mutant problem, higher order mutation, second order mutation
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-6655 (URN)10.1109/TSE.2013.44 (DOI)000334665800003 ()oai:bth.se:forskinfoE56224B7C4384CD5C1257CDD004984E9 (Local ID)oai:bth.se:forskinfoE56224B7C4384CD5C1257CDD004984E9 (Archive number)oai:bth.se:forskinfoE56224B7C4384CD5C1257CDD004984E9 (OAI)
Available from: 2014-07-17 Created: 2014-05-19 Last updated: 2018-01-11Bibliographically approved
Afzal, W., Torkar, R., Feldt, R. & Gorschek, T. (2014). Prediction of faults-slip-through in large software projects: an empirical evaluation. Software quality journal, 22(1), 51-86
Open this publication in new window or tab >>Prediction of faults-slip-through in large software projects: an empirical evaluation
2014 (English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 22, no 1, p. 51-86Article in journal (Refereed) Published
Abstract [en]

A large percentage of the cost of rework can be avoided by finding more faults earlier in a software test process. Therefore, determination of which software test phases to focus improvement work on has considerable industrial interest. We evaluate a number of prediction techniques for predicting the number of faults slipping through to unit, function, integration, and system test phases of a large industrial project. The objective is to quantify improvement potential in different test phases by striving toward finding the faults in the right phase. The results show that a range of techniques are found to be useful in predicting the number of faults slipping through to the four test phases; however, the group of search-based techniques (genetic programming, gene expression programming, artificial immune recognition system, and particle swarm optimization-based artificial neural network) consistently give better predictions, having a representation at all of the test phases. Human predictions are consistently better at two of the four test phases. We conclude that the human predictions regarding the number of faults slipping through to various test phases can be well supported by the use of search-based techniques. A combination of human and an automated search mechanism (such as any of the search-based techniques) has the potential to provide improved prediction results.

Place, publisher, year, edition, pages
Springer, 2014
Keywords
Empirical, Faults-slip-through, Prediction, Search-based
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-6722 (URN)10.1007/s11219-013-9205-3 (DOI)000331643500005 ()oai:bth.se:forskinfo3D40224F7CBF862DC1257B7800251E66 (Local ID)oai:bth.se:forskinfo3D40224F7CBF862DC1257B7800251E66 (Archive number)oai:bth.se:forskinfo3D40224F7CBF862DC1257B7800251E66 (OAI)
Available from: 2014-04-23 Created: 2013-05-27 Last updated: 2018-01-11Bibliographically approved
Organisations

Search in DiVA

Show all publications