Context: Traceability is a key quality attribute of artifacts that are used in knowledge-intensive tasks and supports software engineers in producing higher-quality software. Despite its clear benefits, traceability is often neglected in practice due to challenges such as granularity of traces, lack of a common artifact structure, and unclear responsibility. The Taxonomic Trace Links (TTL) approach connects source and target artifacts through a domain-specific taxonomy, aiming to address these common traceability challenges.

Objective: In this study, we empirically evaluate TTL in an industrial setting to identify its strengths and weaknesses for real-world adoption.

Method: We conducted a mixed-methods study at Ericsson involving one of its software products. Quantitative and qualitative data were collected across two traceability use cases. We established trace links between 463 business use cases, 64 test cases, and 277 ISO-standard requirements. Additionally, we held three focus group sessions with practitioners.

Results: We identified two practically relevant scenarios where traceability is required and evaluated TTL in each. Overall, practitioners found TTL to be a useful solution for identifying trace links with reasonable effort. However, developing a domain-specific taxonomy and managing heterogeneous artifact structures were noted as significant challenges.

Conclusion: TTL is a promising approach that can be adopted in practice and enables traceability use cases. However, TTL are not a replacement for traditional trace links, but complementary to enable more traceability use cases, and encourage early trace links creation.

Background: Poor communication of requirements between clients and suppliers contributes to project overruns, in both software and infrastructure projects. Existing literature offers limited insights into the communication challenges at this interface.

Aim: Our research aim to explore the processes and associated challenges with requirements activities that include client-supplier interaction and communication.

Method: we study requirements validation, communication, and digital asset verification processes through two case studies in the road and railway sectors, involving interviews with ten experts across three companies.

Results: We identify 13 challenges, along with their causes and consequences, and suggest solution areas from existing literature.

Conclusion: Interestingly, the challenges in infrastructure projects mirror those found in software engineering, highlighting a need for further research to validate potential solutions.

Background:

Multi-label requirements classification is an inherently challenging task, especially when dealing with numerous classes at varying levels of abstraction. The task becomes even more difficult when a limited number of requirements is available to train a supervised classifier.  Zero-shot learning does not require training data and can potentially address this problem.

Objective:

This paper investigates the performance of zero-shot classifiers on a multi-label industrial dataset. The study focuses on classifying requirements according to a hierarchical taxonomy designed to support requirements tracing.

Method:

We compare multiple variants of zero-shot classifiers using different embeddings, including 9 language models (LMs) with a reduced number of parameters (up to 3B), e.g., BERT, and 5 large LMs (LLMs) with a large number of parameters (up to 70B), e.g., Llama. Our ground truth includes 377 requirements and 1968 labels from 6 output spaces. For the evaluation, we adopt traditional metrics, i.e., precision, recall, $F_1$, and $F_\beta$, as well as a novel label distance metric $D_n$. This aims to better capture the classification's hierarchical nature and to provide a more nuanced evaluation of how far the results are from the ground truth.

Results:

1) The top-performing model on 5 out of 6 output spaces is T5-xl, with maximum  $F_\beta = 0.78$ and $D_n = 0.04$, while BERT base outperformed the other models in one case, with maximum $F_\beta = 0.83$ and $D_n = 0.04$. 2) LMs with smaller parameter size produce the best classification results compared to LLMs. Thus, addressing the problem in practice is feasible as limited computing power is needed. 3) The model architecture (autoencoding, autoregression, and sentence-to-sentence) significantly affects the classifier's performance.

Contribution:

We conclude that using zero-shot learning for multi-label requirements classification offers promising results. We also present a novel metric that can be used to select the top-performing model for this problem.

Background: Software engineering is a knowledge-intensive activity that requires engineers to manage information to collaborate efficiently and effectively. Within Software Engineering, the Requirements Engineering process bridges the knowledge gap between the customer and the development team by eliciting, managing, and communicating product requirements. The traceability of these requirements supports developers in producing higher-quality software that aligns with customer needs. In addition, traceability supports other activities, such as change impact analysis, software quality assurance, and requirements-based verification.

Problem: Despite decades of research on traceability, practical challenges still hinder the adoption of traceability in practice. This signals a need for new ways of practicing traceability that fit real-world needs. 

Goal: Building on previous work, this thesis instantiates, develops, and empirically evaluates Taxonomic Trace Links, a new way to trace requirements to various software artifacts through domain knowledge captured in a taxonomy. 

Method: The studies included in this theses follows mixed research methods, which are case study, systematic mapping studies, validation study, controlled experiments, and focus groups.

Results: The current state of practice in customer-supplier communication shows persistent challenges that we mapped to solutions in the literature. Our literature study shows that traceability through domain-specific taxonomies has not been empirically evaluated. Our development and evaluation of the technical solution for taxonomic trace links show that semi-automation of trace link creation and maintenance is possible. Finally, our empirical evaluation of taxonomic trace links shows that the solution is feasible in practice and can create trace links for multiple purposes.

Conclusion: Traceability between software artifacts has more benefits than currently realized by practitioners. However, current traceability solutions, based on direct trace links, do not appear to be easily adapted in different scenarios to trace different artifacts. Taxonomic trace links are an alternative approach that could overcome the shortcomings of direct trace links. 

Requirements attributes play an important role in storing and managing meta-information about requirements. This paper presents the results of a literature review and two industrial case studies performed at two large organizations developing software-intensive products for a global market. We performed seven snowballing iterations and identified 18 studies where we extracted requirements attributes. Next, we compare these identified attributes with those of two large companies developing software-intensive products for a global market. We found common attributes that describe stakeholders and roles, support change management, tracing and communication, tracking the status, and estimating the business value of requirements. 

Context and motivation:

Classification aids software development activities by organizing requirements in classes for easier access and retrieval. The majority of requirements classification research has, so far, focused on binary or multi-class classification.

Question/problem:

Multi-label classification with large taxonomies could aid requirements traceability but is prohibitively costly with supervised training. Hence, we investigate zero-short learning to evaluate the feasibility of multi-label requirements classification with large taxonomies.

Principal ideas/results:

We associated, together with domain experts from the industry, 129 requirements with 769 labels from taxonomies ranging between 250 and 1183 classes. Then, we conducted a controlled experiment to study the impact of the type of classifier, the hierarchy, and the structural characteristics of taxonomies on the classification performance. The results show that: (1) The sentence-based classifier had a significantly higher recall compared to the word-based classifier; however, the precision and F1-score did not improve significantly. (2) The hierarchical classification strategy did not always improve the performance of requirements classification. (3) The total and leaf nodes of the taxonomies have a strong negative correlation with the recall of the hierarchical sentence-based classifier.

Contribution:

We investigate the problem of multi-label requirements classification with large taxonomies, illustrate a systematic process to create a ground truth involving industry participants, and provide an analysis of different classification pipelines using zero-shot learning. © 2024 IEEE.

Introduction: Taxonomies capture knowledge about a particular domain in a succinct manner and establish a common understanding among peers. Researchers use taxonomies to convey information about a particular knowledge area or to support automation tasks, and practitioners use them to enable communication beyond organizational boundaries. Aims: Despite this important role of taxonomies in software engineering, their quality is seldom evaluated. Our aim is to identify and define taxonomy quality attributes that provide practical measurements, helping researchers and practitioners to compare taxonomies and choose the one most adequate for the task at hand. Methods: We reviewed 324 publications from software engineering and information systems research and synthesized, when provided, the definitions of quality attributes and measurements. We evaluated the usefulness of the measurements on six taxonomies from three domains. Results: We propose the definition of seven quality attributes and suggest internal and external measurements that can be used to assess a taxonomy’s quality. For two measurements we provide implementations in Python. We found the measurements useful for deciding which taxonomy is best suited for a particular purpose. Conclusion: While there exist several guidelines for creating taxonomies, there is a lack of actionable criteria to compare taxonomies. In this paper, we fill this gap by synthesizing from a wealth of literature seven, non‐overlapping taxonomy quality attributes and corresponding measurements. Future work encompasses their further evaluation of usefulness and empirical validation.

Model-based testing (MBT) is a method that supports the design and execution of test cases by models that specify theintended behaviors of a system under test. While systematic literature reviews on MBT in general exist, the state of the arton modeling and testing performance requirements has seen much less attention. Therefore, we conducted a systematic map-ping study on model-based performance testing. Then, we studied natural language software requirements specificationsin order to understand which and how performance requirements are typically specified. Since none of the identified MBTtechniques supported a major benefit of modeling, namely identifying faults in requirements specifications, we developed thePerformance Requirements verificatiOn and Test EnvironmentS generaTion approach (PRO-TEST). Finally, we evaluatedPRO-TEST on 149 requirements specifications. We found and analyzed 57 primary studies from the systematic mappingstudy and extracted 50 performance requirements models. However, those models don’t achieve the goals of MBT, whichare validating requirements, ensuring their testability, and generating the minimum required test cases. We analyzed 77 Soft-ware Requirements Specification (SRS) documents, extracted 149 performance requirements from those SRS, and illustratethat with PRO-TEST we can model performance requirements, find issues in those requirements and detect missing ones.We detected three not-quantifiable requirements, 43 not-quantified requirements, and 180 underspecified parameters in the149 modeled performance requirements. Furthermore, we generated 96 test environments from those models. By modelingperformance requirements with PRO-TEST, we can identify issues in the requirements related to their ambiguity, measur-ability, and completeness. Additionally, it allows to generate parameters for test environments

In the taxonomic trace links concept, the source and target artifacts are connected through knowledge organization structure (e.g., taxonomy). We introduce in this paper a recommender system that recommends labels to requirements artifacts from domain-specific taxonomy to establish taxonomic trace links. The tool exploits the hierarchical nature of taxonomies and uses requirements text and context information as input to the recommender. © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

Background Requirements engineering and verification (REV) processes play es-sential roles in software product development. There are physical and non-physicaldistances between entities (actors, artifacts, and activities) in these processes. Cur-rent practices that reduce the distances, such as automated testing and alignmentof document structure and tracing only partially close the above mentioned gap.Objective The aim of this thesis is to investigate solutions w.r.t their abilityto reduce the distances between requirements engineering and verification. Twotechniques that are explored in this thesis are automated testing (model-basedtesting, MBT) and alignment of document structure and tracing (traceability).Method The research methods used in this thesis are systematic mapping, soft-ware requirements mining, case study, literature survey, validation study, and de-sign science.Results MBT and traceability are effective in reducing the distance between re-quirements and verification. However, both activities have some shortcoming thatneeds to be addressed when used for that purpose. Current MBT techniques inthe context of software performance do not attain all the goals of MBT: 1) require-ments validation, 2) checking the testability of requirements, and 3) the generationof an efficient test suite. These goals are essential to reduce the distance. We de-veloped and assessed performance requirements verification and test environmentgeneration approach to tackle these shortcomings. Also, traceability between re-quirements and verification suffers from the low granularity of trace links and doesnot support the verification of all requirements. We propose the use of taxonomictrace links to trace and align the structure of requirements specifications and ver-ification artifacts. The results from the validation study show that the solution isfeasible in practice. However, this comes with challenges that need to be addressed.Conclusion MBT and improved traceability reduce multiple distances betweenactors, artifacts, and activities in the requirements engineering and verificationprocess. MBT is most effective in reducing the distances when the model used isbuilt from the requirements. Traceability is essential in easing access to relevantinformation when needed and should not be seen as an overhead. When creatingtrace links, we need to consider the difference in the abstraction, structure, andtime between the linked artifacts