Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Test Set Diameter: Quantifying the Diversity of Sets of Test Cases
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
UCL, GBR.
Korea Adv Inst Sci & Technol, KOR.
2016 (English)In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST, IEEE Computer Society, 2016, 223-233 p.Conference paper (Refereed)
Resource type
Text
Abstract [en]

A common and natural intuition among software testers is that test cases need to differ if a software system is to be tested properly and its quality ensured. Consequently, much research has gone into formulating distance measures for how test cases, their inputs and/or their outputs differ. However, common to these proposals is that they are data type specific and/or calculate the diversity only between pairs of test inputs, traces or outputs. We propose a new metric to measure the diversity of sets of tests: the test set diameter (TSDm). It extends our earlier, pairwise test diversity metrics based on recent advances in information theory regarding the calculation of the normalized compression distance (NCD) for multisets. A key advantage is that TSDm is a universal measure of diversity and so can be applied to any test set regardless of data type of the test inputs (and, moreover, to other test-related data such as execution traces). But this universality comes at the cost of greater computational effort compared to competing approaches. Our experiments on four different systems show that the test set diameter can help select test sets with higher structural and fault coverage than random selection even when only applied to test inputs. This can enable early test design and selection, prior to even having a software system to test, and complement other types of test automation and analysis. We argue that this quantification of test set diversity creates a number of opportunities to better understand software quality and provides practical ways to increase it.

Place, publisher, year, edition, pages
IEEE Computer Society, 2016. 223-233 p.
Series
IEEE International Conference on Software Testing Verification and Validation, ISSN 2381-2834
Keyword [en]
Empirical study; Information theory; Software testing; Test selection
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-11201DOI: 10.1109/ICST.2016.33ISI: 000391252900021ISBN: 9781509018260 (print)OAI: oai:DiVA.org:bth-11201DiVA: diva2:894204
Conference
9th IEEE International Conference on Software Testing, Verification and Validation, ICST 2016; Chicago
Available from: 2016-01-14 Created: 2015-12-14 Last updated: 2017-02-03Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Feldt, RobertPoulding, Simon M.
By organisation
Department of Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

Altmetric score

Total: 74 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf