Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Assessing Reusability in Automated Acceptance Tests
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
2018 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Context: Automated acceptance tests have become a core practice of agile software development (e.g. Extreme Programming). These tests are closely tied to requirements specifications and these tests provide a mechanism for continuous validation of software requirements. Software reuse has evolved with the introduction of each new reusable artefact (e.g., reuse of code, reuse of frameworks, tools etc.). In this study, we have investigated the reusability of automated acceptance tests keeping in view their close association with textual requirements.

Objective: As automated acceptance tests are closely related to software requirements, we have used existing research in software engineering to identify reusability related characteristics of software requirements and used these characteristics for automated acceptance tests.This study attempts to address the following aspects: (i) what important reuse characteristics should be considered when measuring reusability of automated acceptance tests? (ii) how reusability can be measured in automated acceptance tests?, and (iii) how cost avoided through reuse of automated acceptance tests can be calculated?

Method: We have used a combination of research methods to answer different aspects of our study. We started by identifying reusability related characteristics of software requirements, with help of systematic literature review. Later, we tried to identify the reusability related characteristics of defect reports and the process is documented using an experience report. After identifying the characteristics from the previous two studies, we used these characteristics on two case-studies conducted on Behaviour driven development test cases (i.e., acceptance tests of textual nature). We proposed two approaches that can identify the reuse potential of automated acceptance tests and evaluated these approaches in the industry. Later, to calculate the cost avoided through reuse, we proposed and evaluated a method that is applicable to any reusable artifact.

Results: The results from the systematic literature review shows that text-based requirements reuse approaches are most commonly used in the industry. Structuring these text-based requirements and identifying the reusable requirements by matching are the two commonly used methods for enabling requirements to reuse. The results from the second study, industrial experience report, indicates that defect reports can be formulated in template and defect triage meeting can be used to identify important test-cases related to defect reports. The results from these two studies, text-based requirements reuse approaches and template based defect reports, were included when identifying approaches to measure reuse potential of BDD test-cases. The two proposed approaches, Normalised Compression Distance (NCD) and Similarity Ratio, for measuring reuse potential were evaluated in the industry. The evaluation indicated that Similarity ratio approach performed better than the NCD approach, however, the results from both approaches were comparable with the results gathered with the help of expert analysis. The cost related aspects of reusable acceptance tests were addressed and evaluated using a method that calculates the cost-avoidance through reuse. The industrial evaluation of the method and guidelines show that the method is an artifact independent method. 

Conclusions: The evidence from this study shows that the automated acceptance tests are reusable, similar to text-based software requirements and their reuse potential can be calculated as well. The industrial evaluation of the three studies (i.e. approaches to measure reuse potential, calculation of cost avoidance and defect reports in triage meetings) shows that the overall results are applicable to the industry. However, further work is required to evaluate the reuse potential of automated acceptance tests in different contexts. 

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2018. , p. 183
Series
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 1
National Category
Other Engineering and Technologies not elsewhere specified
Identifiers
URN: urn:nbn:se:bth-15633ISBN: 978-91-7295-347-5 (print)OAI: oai:DiVA.org:bth-15633DiVA, id: diva2:1164723
Available from: 2017-12-12 Created: 2017-12-11 Last updated: 2018-02-02Bibliographically approved
List of papers
1. A systematic literature review of software requirements reuse approaches
Open this publication in new window or tab >>A systematic literature review of software requirements reuse approaches
2018 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 93, no Jan, p. 223-245Article in journal (Refereed) Published
Abstract [en]

Context: Early software reuse is considered as the most beneficial form of software reuse. Hence, previous research has focused on supporting the reuse of software requirements. Objective: This study aims to identify and investigate the current state of the art with respect to (a) what requirement reuse approaches have been proposed, (b) the methods used to evaluate the approaches, (c) the characteristics of the approaches, and (d) the quality of empirical studies on requirements reuse with respect to rigor and relevance. Method: We conducted a systematic review and a combination of snowball sampling and database search have been used to identify the studies. The rigor and relevance scoring rubric has been used to assess the quality of the empirical studies. Multiple researchers have been involved in each step to increase the reliability of the study. Results: Sixty-nine studies were identified that describe requirements reuse approaches. The majority of the approaches used structuring and matching of requirements as a method to support requirements reuse and text-based artefacts were commonly used as an input to these approaches. Further evaluation of the studies revealed that the majority of the approaches are not validated in the industry. The subset of empirical studies (22 in total) was analyzed for rigor and relevance and two studies achieved the maximum score for rigor and relevance based on the rubric. It was found that mostly text-based requirements reuse approaches were validated in the industry. Conclusion: From the review, it was found that a number of approaches already exist in literature, but many approaches are not validated in industry. The evaluation of rigor and relevance of empirical studies show that these do not contain details of context, validity threats, and the industrial settings, thus highlighting the need for the industrial evaluation of the approaches. © 2017 Elsevier B.V.

Place, publisher, year, edition, pages
Elsevier, 2018
Keywords
Artefact reuse, Relevance, Requirements reuse, Reusability, Rigor, Software requirements, Quality control, Requirements engineering, Search engines, Computer software reusability
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-15356 (URN)10.1016/j.infsof.2017.09.009 (DOI)000414878200014 ()2-s2.0-85030849171 (Scopus ID)
Funder
Knowledge Foundation
Note

project "Professional Licentiate of Engineering Research School"

Available from: 2017-10-20 Created: 2017-10-20 Last updated: 2018-01-13Bibliographically approved
2. Capturing cost avoidance through reuse: Systematic literature review and industrial evaluation
Open this publication in new window or tab >>Capturing cost avoidance through reuse: Systematic literature review and industrial evaluation
2016 (English)In: ACM International Conference Proceeding Series, ACM Press, 2016, Vol. 01-03-June-2016Conference paper, Published paper (Refereed)
Abstract [en]

Background: Cost avoidance through reuse shows the benefits gained by the software organisations when reusing an artefact. Cost avoidance captures benefits that are not captured by cost savings e.g. spending that would have increased in the absence of the cost avoidance activity. This type of benefit can be combined with quality aspects of the product e.g. costs avoided because of defect prevention. Cost avoidance is a key driver for software reuse. Objectives: The main objectives of this study are: (1) To assess the status of capturing cost avoidance through reuse in the academia; (2) Based on the first objective, propose improvements in capturing of reuse cost avoidance, integrate these into an instrument, and evaluate the instrument in the software industry. Method: The study starts with a systematic literature review (SLR) on capturing of cost avoidance through reuse. Later, a solution is proposed and evaluated in the industry to address the shortcomings identified during the systematic literature review. Results: The results of a systematic literature review describe three previous studies on reuse cost avoidance and show that no solution, to capture reuse cost avoidance, was validated in industry. Afterwards, an instrument and a data collection form are proposed that can be used to capture the cost avoided by reusing any type of reuse artefact. The instrument and data collection form (describing guidelines) were demonstrated to a focus group, as part of static evaluation. Based on the feedback, the instrument was updated and evaluated in industry at 6 development sites, in 3 different countries, covering 24 projects in total. Conclusion: The proposed solution performed well in industrial evaluation. With this solution, practitioners were able to do calculations for reuse costs avoidance and use the results as decision support for identifying potential artefacts to reuse.

Place, publisher, year, edition, pages
ACM Press, 2016
Keywords
Costs; Data acquisition; Decision support systems; Software engineering, Cost avoidances; Cost saving; Decision supports; Defect prevention; Industrial evaluations; Software organisations; Systematic literature review; Systematic literature review (SLR), Computer software reusability
National Category
Software Engineering
Identifiers
urn:nbn:se:bth-13116 (URN)10.1145/2915970.2915989 (DOI)2-s2.0-84978540065 (Scopus ID)9781450336918 (ISBN)
Conference
20th International Conference on Evaluation and Assessment in Software Engineering, EASE 2016; Limerick
Available from: 2016-10-04 Created: 2016-10-03 Last updated: 2018-01-14Bibliographically approved

Open Access in DiVA

fulltext(9481 kB)83 downloads
File information
File name FULLTEXT02.pdfFile size 9481 kBChecksum SHA-512
948b85a908d69198f1ae19760ab352076298dc81ad826160e15fcb61be9d78c483cff3be50ed880388036980a4245fefe4baf91934e60314dec91721e240ea74
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Irshad, Mohsin
By organisation
Department of Software Engineering
Other Engineering and Technologies not elsewhere specified

Search outside of DiVA

GoogleGoogle Scholar
Total: 128 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 310 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf