Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Utilising CI environment for efficient and effective testing of NFRs
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Qvantel Sweden AB.ORCID iD: 0000-0001-5949-1375
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
Örebro universitet, SWE.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
2020 (English)In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 117, article id 106199Article in journal (Refereed) Published
Abstract [en]

Context: Continuous integration (CI) is a practice that aims to continuously verify quality aspects of a software intensive system both for functional and non-functional requirements (NFRs). Functional requirements are the inputs of development and can be tested in isolation, utilising either manual or automated tests. In contrast, some NFRs are difficult to test without functionality, for NFRs are often aspects of functionality and express quality aspects. Lacking this testability attribute makes NFR testing complicated and, therefore, underrepresented in industrial practice. However, the emergence of CI has radically affected software development and created new avenues for software quality evaluation and quality information acquisition. Research has, consequently, been devoted to the utilisation of this additional information for more efficient and effective NFR verification. Objective: We aim to identify the state-of-the-art of utilising the CI environment for NFR testing, hereinafter referred to as CI-NFR testing. Method: Through rigorous selection, from an initial set of 747 papers, we identified 47 papers that describe how NFRs are tested in a CI environment. Evidence-based analysis, through coding, is performed on the identified papers in this SLR. Results: Firstly, ten CI approaches are described by the papers selected, each describing different tools and nine different NFRs where reported to be tested. Secondly, although possible, CI-NFR testing is associated with eight challenges that adversely affect its adoption. Thirdly, the identified CI-NFR testing processes are tool-driven, but there is a lack of NFR testing tools that can be used in the CI environment. Finally, we proposed a CI framework for NFRs testing. Conclusion: A synthesised CI framework is proposed for testing various NFRs, and associated CI tools are also mapped. This contribution is valuable as results of the study also show that CI-NFR testing can help improve the quality of NFR testing in practices. © 2019

Place, publisher, year, edition, pages
Elsevier B.V. , 2020. Vol. 117, article id 106199
Keywords [en]
Agile, CI, Continuous integration, DevOps, NFR, Non-functional requirement, SAFe, Scaled agile framework, Computer software selection and evaluation, Integral equations, Paper, Software design, Continuous integrations, Non-functional requirements, Quality control
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-18829DOI: 10.1016/j.infsof.2019.106199ISI: 000496874400005Scopus ID: 2-s2.0-85073572821OAI: oai:DiVA.org:bth-18829DiVA, id: diva2:1367037
Available from: 2019-10-31 Created: 2019-10-31 Last updated: 2023-06-02Bibliographically approved
In thesis
1. Utilizing Continuous Integration environments for evaluation of software quality attributes
Open this publication in new window or tab >>Utilizing Continuous Integration environments for evaluation of software quality attributes
2023 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Software quality attributes are properties that reflect the quality of a software system, and Non-functional requirements (NFRs) are the specifications that define how a software system should perform to reach a desired level of goals of the quality attributes.The evaluation of quality attributes is important to show the effectiveness of a system in meeting customers' NFRs.

Continuous integration (CI) environments have emerged as powerful platforms for organizations to improve software quality through automated software verification and validation.Despite this, there is a growing need for evaluating quality attributes that is often met by in-house development of metrics and tools.This highlights the importance of quality attributes for software product quality.

This thesis investigates the association between quality attributes and components of a CI environment, as well as how to utilize these components for evaluating software quality attributes.The focus is on improving the knowledge of the evaluation and providing specific recommendations for companies to enhance their CI environments for higher demands of quality evaluation.The contributions of this thesis include a better understanding of the relationship between quality attributes and CI components, and a set of practical guidelines for companies to effectively leverage CI for quality attribute evaluation.

The studies in this thesis utilized mixed methodologies, including a systematic literature review, a multi-case study conducted in four software development companies, and an synthesis of the collected data.The multi-case study provided a comprehensive overview of practices for quality attribute evaluation and how CI components can generate data to support the evaluation of specific attributes.The synthesis study presents a maturity model based on the collected data from both academia and industry, and the model can aid organizations in assessing their current level of maturity in utilizing CI environments to identify potential improvements.The results in these studies show the capabilities of different components of a CI environment and how these components can be used to support the evaluation of quality attributes.While the use of CI environments for the thesis topic offers benefits, it also presents several challenges, for example, the challenge to identify effective quality metrics.

In conclusion, this thesis contributes to the understanding of the use of CI environments for evaluating software quality attributes.The results suggest that CI environments can be an effective approach for quality attribute evaluation, but suitable metrics need to be considered to ensure accurate and meaningful evaluation results. Furthermore, the thesis presents areas for future research, such as the use of machine learning techniques to improve the accuracy of quality assessment using CI environments.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2023
Series
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 5
Keywords
Quality attributes, Quality metrics, Non-functional requirement, Continuous integration, Empirical software engineering
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-24402 (URN)978-91-7295-457-1 (ISBN)
Presentation
2023-06-14, J1630 + Zoom, Karlskrona, 13:00 (English)
Opponent
Supervisors
Available from: 2023-04-17 Created: 2023-04-12 Last updated: 2023-08-22Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Yu, LiangAlégroth, EmilChatzipetrou, PanagiotaGorschek, Tony

Search in DiVA

By author/editor
Yu, LiangAlégroth, EmilChatzipetrou, PanagiotaGorschek, Tony
By organisation
Department of Software Engineering
In the same journal
Information and Software Technology
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 535 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf