CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Automated NFR testing in continuous integration environments: a multi-case study of Nordic companies
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-5949-1375
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0001-7526-3727
Örebro University School of Business.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.ORCID iD: 0000-0002-3646-235x
2023 (English)In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 28, no 6, article id 144Article in journal (Refereed) Published
Abstract [en]

Context: Non-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software. Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation. Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of Continuous Integration (CI). However, studies on how companies manage automated NFR testing through CI are limited. Objective: This study examines how automated NFR testing can be enabled and supported using CI environments in software development companies. Method: We performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners. Results: Maintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments. Testing practices, quality metrics, and challenges for measuring NFRs were reported. Conclusions: This study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice. © 2023, The Author(s).

Place, publisher, year, edition, pages
Springer, 2023. Vol. 28, no 6, article id 144
Keywords [en]
Automated testing, Case study, CI, Continuous integration, Metrics, NFR, Non-functional requirements, Automation, Integration, Integration testing, Quality control, Software design, Case-studies, Continuous integrations, Integration environments, Metric, Nordic companies, System quality, Quality assurance
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-25556DOI: 10.1007/s10664-023-10356-1ISI: 001087927600001Scopus ID: 2-s2.0-85174862814OAI: oai:DiVA.org:bth-25556DiVA, id: diva2:1809787
Part of project
SERT- Software Engineering ReThought, Knowledge FoundationPLEng – Professional Licentiate of Engineering School, Knowledge Foundation
Funder
Knowledge Foundation, 20180010Knowledge Foundation, 20170213Available from: 2023-11-06 Created: 2023-11-06 Last updated: 2024-01-02Bibliographically approved

Open Access in DiVA

fulltext(1692 kB)59 downloads
File information
File name FULLTEXT01.pdfFile size 1692 kBChecksum SHA-512
dfc4f54719213f2a47972939ffe0b305a11115e67e0126f0f96fbc5831ae9526d9cf69c160b07eb0d8694ba8a73542bfbd18f2c3fdbb68ade1673f3fc81522d3
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Yu, LiangAlégroth, EmilGorschek, Tony

Search in DiVA

By author/editor
Yu, LiangAlégroth, EmilGorschek, Tony
By organisation
Department of Software Engineering
In the same journal
Empirical Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 60 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 210 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf