Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Structuring Exploratory Testing through Test Charter Design and Decision Support
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik. (SERL)ORCID-id: 0000-0001-9336-4361
2017 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Context: Exploratory testing (ET) is an approach to test software with a strong focus on personal skills and freedom of the tester. ET emphasises the simultaneous design and execution of tests with minimal test documentation. Test practitioners often claim that their choice to use ET as an important alternative to scripted testing is based on several benefits ET exhibits over the scripted testing. However, these claims lack empirical evidence as there is little research done in this area. Moreover, ET is usually considered an ad-hoc way of doing testing as everyone does it differently. There have been some attempts in past to provide structure to ET. Session based test management (SBTM) is an approach that attempts to provide some structure to ET and gives some basic guidelines to structuring the test sessions. However, these guidelines are still very abstract and are very open to individuals' interpretation.

Objective: The main objective of this doctoral thesis is to support practitioners in their decisions about choosing exploratory versus scripted testing. Furthermore, it is also aimed to investigate the empirical evidence in support of ET and find ways to structure ET and classify different levels of exploration that drive the choices made by exploratory testers. Another objective of this thesis is to provide a decision support system to select levels of exploration in overall test process.

Method: The findings presented in this thesis are obtained through a controlled experiment with participants from industry and academia, exploratory surveys, interviews and focus groups conducted at different companies including Ericsson AB, Sony Mobile Communications, Axis Communications AB and Softhouse Consulting Baltic AB.

Results: Using the exploratory survey, we found three test techniques to be most relevant in context of testing software systems and in particular heterogeneous systems. The most frequently used technique mentioned by the practitioners is ET which is not a much researched topic. We also found many interesting claims about ET in grey literature produced by practitioners in the form of informal presentations and blogs but these claims lacked any empirical evidence. Therefore, a controlled experiment was conducted with students and industry practitioners to compare ET with scripted testing. The experiment results show that ET detects significantly more critical defects compared to scripted testing and is more time efficient. However, ET has its own limitations and there is not a single way to use it for testing. In order to provide structure to ET, we conducted a study where we propose checklists to support test charter design in ET. Furthermore, two more industrial focus group studies at four companies were conducted that resulted in a taxonomy of exploration levels in ET and a decision support method for selecting exploration levels in ET. Lastly, we investigated different problems that researchers face when conducting surveys in software engineering and have presented mitigation strategies for these problems.

Conclusion: The taxonomy for levels of exploration in ET, proposed in this thesis, provided test practitioners at the companies a better understanding of the underlying concepts of ET and a way to structure their test charters. A number of influence factors elicited as part of this thesis also help them prioritise which level of exploration suits more to their testing in the context of their products. Furthermore, the decision support method provided the practitioners to reconsider their current test focus to test their products in a more effective way.

Ort, förlag, år, upplaga, sidor
Karlskrona: Blekinge Tekniska Högskola, 2017.
Serie
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 6
Nyckelord [en]
Exploratory Testing, Software Testing, Test Charter Design, Decision Support, Survey Research, Software Engineering, Session based test management, SBTM, ET
Nationell ämneskategori
Programvaruteknik
Identifikatorer
URN: urn:nbn:se:bth-14121ISBN: 978-91-7295-339-0 (tryckt)OAI: oai:DiVA.org:bth-14121DiVA, id: diva2:1089389
Disputation
2017-06-01, J1630, Campus Gräsvik, Valhalavägen 1, Karlskrona, 14:37 (Engelska)
Opponent
Handledare
Tillgänglig från: 2017-05-18 Skapad: 2017-04-19 Senast uppdaterad: 2018-01-13Bibliografiskt granskad
Delarbeten
1. Heterogeneous Systems Testing Techniques: An Exploratory Survey
Öppna denna publikation i ny flik eller fönster >>Heterogeneous Systems Testing Techniques: An Exploratory Survey
2015 (Engelska)Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Heterogeneous systems comprising sets of inherent subsystems are challenging to integrate. In particular, testing for interoperability and conformance is a challenge. Furthermore, the complexities of such systems amplify traditional testing challenges. We explore (1) which techniques are frequently discussed in literature in context of heterogeneous system testing that practitioners use to test their heterogeneous systems; (2) the perception of the practitioners on the usefulness of the techniques with respect to a defined set of outcome variables. For that, we conducted an exploratory survey. A total of 27 complete survey answers have been received. Search-based testing has been used by 14 out of 27 respondents, indicating the practical relevance of the approach for testing heterogeneous systems, which itself is relatively new and has only recently been studied extensively. The most frequently used technique is exploratory manual testing, followed by combinatorial testing. With respect to the perceived performance of the testing techniques, the practitioners were undecided regarding many of the studied variables. Manual exploratory testing received very positive ratings across outcome variables.

Ort, förlag, år, upplaga, sidor
Vienna: Springer Berlin Heidelberg, 2015
Serie
Lecture Notes in Business Information Processing, ISSN 1865-1348
Nyckelord
Heterogeneous systems, software testing
Nationell ämneskategori
Programvaruteknik
Identifikatorer
urn:nbn:se:bth-6433 (URN)10.1007/978-3-319-13251-8_5 (DOI)oai:bth.se:forskinfo148BC605AB9D74B6C1257D4200420D1D (Lokalt ID)oai:bth.se:forskinfo148BC605AB9D74B6C1257D4200420D1D (Arkivnummer)oai:bth.se:forskinfo148BC605AB9D74B6C1257D4200420D1D (OAI)
Externt samarbete:
Konferens
7th International Conference on Software Quality Days, SWQD 2015; Vienna; Austria
Tillgänglig från: 2015-01-12 Skapad: 2014-08-28 Senast uppdaterad: 2018-01-11Bibliografiskt granskad
2. An experiment on the effectiveness and efficiency of exploratory testing
Öppna denna publikation i ny flik eller fönster >>An experiment on the effectiveness and efficiency of exploratory testing
Visa övriga...
2015 (Engelska)Ingår i: Empirical Software Engineering, ISSN 1382-3256, Vol. 20, nr 3, s. 844-878Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The exploratory testing (ET) approach is commonly applied in industry, but lacks scientific research. The scientific community needs quantitative results on the performance of ET taken from realistic experimental settings. The objective of this paper is to quantify the effectiveness and efficiency of ET vs. testing with documented test cases (test case based testing, TCT). We performed four controlled experiments where a total of 24 practitioners and 46 students performed manual functional testing using ET and TCT. We measured the number of identified defects in the 90-minute testing sessions, the detection difficulty, severity and types of the detected defects, and the number of false defect reports. The results show that ET found a significantly greater number of defects. ET also found significantly more defects of varying levels of difficulty, types and severity levels. However, the two testing approaches did not differ significantly in terms of the number of false defect reports submitted. We conclude that ET was more efficient than TCT in our experiment. ET was also more effective than TCT when detection difficulty, type of defects and severity levels are considered. The two approaches are comparable when it comes to the number of false defect reports submitted.

Ort, förlag, år, upplaga, sidor
Springer, 2015
Nyckelord
Software testing, Experiment, Exploratory testing, Efficiency, Effectiveness
Nationell ämneskategori
Programvaruteknik
Identifikatorer
urn:nbn:se:bth-6727 (URN)10.1007/s10664-014-9301-4 (DOI)000354480800008 ()oai:bth.se:forskinfoAF25267D19C21440C1257CBF007311E9 (Lokalt ID)oai:bth.se:forskinfoAF25267D19C21440C1257CBF007311E9 (Arkivnummer)oai:bth.se:forskinfoAF25267D19C21440C1257CBF007311E9 (OAI)
Tillgänglig från: 2014-04-22 Skapad: 2014-04-19 Senast uppdaterad: 2018-01-11Bibliografiskt granskad

Open Access i DiVA

fulltext(12884 kB)1482 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 12884 kBChecksumma SHA-512
6924a18db47126bb91a9ebc22cb01a751e95d31170fd2461fd2b6c630168d1799a3771693fa968ff074385b14b19996333cf8ed0a1b716c55be2621cbb889e9e
Typ fulltextMimetyp application/pdf

Personposter BETA

Ghazi, Ahmad Nauman

Sök vidare i DiVA

Av författaren/redaktören
Ghazi, Ahmad Nauman
Av organisationen
Institutionen för programvaruteknik
Programvaruteknik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 1675 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 1426 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf