Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Automated NFR testing in Continuous Integration Environments: a multi-case study of Nordic companies
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Ericsson AB.ORCID iD: 0000-0001-5949-1375
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-7526-3727
Örebro University.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0002-3646-235x
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Context: Non-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software.Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation.Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of  Continuous Integration (CI).However, studies on how companies manage automated NFR testing through CI are limited.

Objective: This study examines how automated NFR testing can be enabledand supported using CI environments in software development companies.

Method: We performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners.

Results: Maintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments.Testing practices, quality metrics, and challenges for measuring NFRs were reported.

Conclusions: This study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice.

National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-24400OAI: oai:DiVA.org:bth-24400DiVA, id: diva2:1746896
Available from: 2023-03-29 Created: 2023-03-29 Last updated: 2023-08-11Bibliographically approved
In thesis
1. Utilizing Continuous Integration environments for evaluation of software quality attributes
Open this publication in new window or tab >>Utilizing Continuous Integration environments for evaluation of software quality attributes
2023 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Software quality attributes are properties that reflect the quality of a software system, and Non-functional requirements (NFRs) are the specifications that define how a software system should perform to reach a desired level of goals of the quality attributes.The evaluation of quality attributes is important to show the effectiveness of a system in meeting customers' NFRs.

Continuous integration (CI) environments have emerged as powerful platforms for organizations to improve software quality through automated software verification and validation.Despite this, there is a growing need for evaluating quality attributes that is often met by in-house development of metrics and tools.This highlights the importance of quality attributes for software product quality.

This thesis investigates the association between quality attributes and components of a CI environment, as well as how to utilize these components for evaluating software quality attributes.The focus is on improving the knowledge of the evaluation and providing specific recommendations for companies to enhance their CI environments for higher demands of quality evaluation.The contributions of this thesis include a better understanding of the relationship between quality attributes and CI components, and a set of practical guidelines for companies to effectively leverage CI for quality attribute evaluation.

The studies in this thesis utilized mixed methodologies, including a systematic literature review, a multi-case study conducted in four software development companies, and an synthesis of the collected data.The multi-case study provided a comprehensive overview of practices for quality attribute evaluation and how CI components can generate data to support the evaluation of specific attributes.The synthesis study presents a maturity model based on the collected data from both academia and industry, and the model can aid organizations in assessing their current level of maturity in utilizing CI environments to identify potential improvements.The results in these studies show the capabilities of different components of a CI environment and how these components can be used to support the evaluation of quality attributes.While the use of CI environments for the thesis topic offers benefits, it also presents several challenges, for example, the challenge to identify effective quality metrics.

In conclusion, this thesis contributes to the understanding of the use of CI environments for evaluating software quality attributes.The results suggest that CI environments can be an effective approach for quality attribute evaluation, but suitable metrics need to be considered to ensure accurate and meaningful evaluation results. Furthermore, the thesis presents areas for future research, such as the use of machine learning techniques to improve the accuracy of quality assessment using CI environments.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2023
Series
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 5
Keywords
Quality attributes, Quality metrics, Non-functional requirement, Continuous integration, Empirical software engineering
National Category
Software Engineering
Research subject
Software Engineering
Identifiers
urn:nbn:se:bth-24402 (URN)978-91-7295-457-1 (ISBN)
Presentation
2023-06-14, J1630 + Zoom, Karlskrona, 13:00 (English)
Opponent
Supervisors
Available from: 2023-04-17 Created: 2023-04-12 Last updated: 2023-08-22Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Yu, LiangAlégroth, EmilChatzipetrou, PanagiotaGorschek, Tony

Search in DiVA

By author/editor
Yu, LiangAlégroth, EmilChatzipetrou, PanagiotaGorschek, Tony
By organisation
Department of Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 77 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf