Change search
Refine search result
1 - 45 of 45
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Ali, Nauman Bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Is effectiveness sufficient to choose an intervention?: Considering resource use in empirical software engineering2016In: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2016, Ciudad Real, Spain, September 8-9, 2016, 2016, article id 54Conference paper (Refereed)
  • 2.
    Ali, Nauman bin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Edison, Henry
    Lero - The Irish Software Engineering Research Centre, IRL.
    Torkar, Richard
    Chalmers and University of Gothenburg, SWE.
    The impact of a proposal for innovation measurement in the software industry2020In: International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, 2020, article id 3422163Conference paper (Refereed)
    Abstract [en]

    Background: Measuring an organization's capability to innovate and assessing its innovation output and performance is a challenging task. Previously, a comprehensive model and a suite of measurements to support this task were proposed. Aims: In the current paper, seven years since the publication of the paper titled Towards innovation measurement in the software industry, we have reflected on the impact of thework. Method:We have mainly relied on quantitative and qualitative analysis of the citations of the paper using an established classification schema. Results: We found that the article has had a significant scientific impact (indicated by the number of citations), i.e., (1) cited in literature from both software engineering and other fields, (2) cited in grey literature and peerreviewed literature, and (3) substantial citations in literature not published in the English language. However, we consider a majority of the citations in the peer-reviewed literature (75 out of 116) as neutral, i.e., they have not used the innovation measurement paper in any substantial way. All in all, 38 out of 116 have used, modified or based their work on the definitions, measurements or the model proposed in the article. This analysis revealed a significant weakness of the citing work, i.e., among the citing papers, we found only two explicit comparisons to the innovation measurement proposal, and we found no papers that identify weaknesses of said proposal. Conclusions: This work highlights the need for being cautious of relying solely on the number of citations for understanding impact, and the need for further improving and supporting the peer-review process to identify unwarranted citations in papers. © 2020 IEEE Computer Society. All rights reserved.

    Download full text (pdf)
    fulltext
  • 3.
    Ali, Nauman bin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Engström, Emelie
    Lund University, SWE.
    Taromirad, Masoumeh
    Halmstad University, SWE.
    Mousavi, Muhammad Raza
    Halmstad University, SWE.
    Minhas, Nasir Mehmood
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Helgesson, Daniel
    Lund University, SWE.
    Kunze, Sebastian
    Halmstad University, SWE.
    Varshosaz, Mahsa
    Halmstad University, SWE.
    On the search for industry-relevant regression testing research2019In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 24, no 4, p. 2020-2055Article in journal (Refereed)
    Abstract [en]

    Regression testing is a means to assure that a change in the software, or

    its execution environment, does not introduce new defects. It involves the expensive

    undertaking of rerunning test cases. Several techniques have been proposed

    to reduce the number of test cases to execute in regression testing, however, there

    is no research on how to assess industrial relevance and applicability of such techniques.

    We conducted a systematic literature review with the following two goals:

    rstly, to enable researchers to design and present regression testing research with

    a focus on industrial relevance and applicability and secondly, to facilitate the industrial

    adoption of such research by addressing the attributes of concern from the

    practitioners' perspective. Using a reference-based search approach, we identied

    1068 papers on regression testing. We then reduced the scope to only include papers

    with explicit discussions about relevance and applicability (i.e. mainly studies

    involving industrial stakeholders). Uniquely in this literature review, practitioners

    were consulted at several steps to increase the likelihood of achieving our aim of

    identifying factors important for relevance and applicability. We have summarised

    the results of these consultations and an analysis of the literature in three taxonomies,

    which capture aspects of industrial-relevance regarding the regression

    testing techniques. Based on these taxonomies, we mapped 38 papers reporting

    the evaluation of 26 regression testing techniques in industrial settings.

    Download full text (pdf)
    fulltext
  • 4.
    Ali, Nauman Bin
    et al.
    Blekinge Institute of Technology, School of Computing.
    Petersen, Kai
    Blekinge Institute of Technology, School of Computing.
    A consolidated process for software process simulation: State of the Art and Industry Experience2012Conference paper (Refereed)
    Abstract [en]

    Software process simulation is a complex task and in order to conduct a simulation project practitioners require support through a process for software process simulation modelling (SPSM), including what steps to take and what guidelines to follow in each step. This paper provides a literature based consolidated process for SPSM where the steps and guidelines for each step are identified through a review of literature and are complemented by experience from using these recommendations in an action research at a large Telecommunication vendor. We found five simulation processes in SPSM literature, resulting in a seven-step process. The consolidated process was successfully applied at the studied company, with the experiences of doing so being reported.

    Download full text (pdf)
    fulltext
  • 5.
    Ali, Nauman bin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    FLOW-assisted value stream mapping in the early phases of large-scale software development2016In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 111, p. 213-227Article in journal (Refereed)
    Abstract [en]

    Value stream mapping (VSM) has been successfully applied in the context of software process improvement. However, its current adaptations from Lean manufacturing focus mostly on the flow of artifacts and have taken no account of the essential information flows in software development. A solution specifically targeted toward information flow elicitation and modeling is FLOW. This paper aims to propose and evaluate the combination of VSM and FLOW to identify and alleviate information and communication related challenges in large-scale software development. Using case study research, FLOW-assisted VSM was used for a large product at Ericsson AB, Sweden. Both the process and the outcome of FLOW-assisted VSM have been evaluated from the practitioners’ perspective. It was noted that FLOW helped to systematically identify challenges and improvements related to information flow. Practitioners responded favorably to the use of VSM and FLOW, acknowledged the realistic nature and impact on the improvement on software quality, and found the overview of the entire process using the FLOW notation very useful. The combination of FLOW and VSM presented in this study was successful in systematically uncovering issues and characterizing their solutions, indicating their practical usefulness for waste removal with a focus on information flow related issues.

    Download full text (pdf)
    fulltext
  • 6.
    Ali, Nauman Bin
    et al.
    Blekinge Institute of Technology, School of Computing.
    Petersen, Kai
    Blekinge Institute of Technology, School of Computing.
    Mäntylä, Mika
    Testing highly complex system of systems: An industrial case study2012Conference paper (Refereed)
    Abstract [en]

    Systems of systems (SoS) are highly complex and are integrated on multiple levels (unit, component, system, system of systems). Many of the characteristics of SoS (such as operational and managerial independence, integration of system into system of systems, SoS comprised of complex systems) make their development and testing challenging. Contribution: This paper provides an understanding of SoS testing in large-scale industry settings with respect to challenges and how to address them. Method: The research method used is case study research. As data collection methods we used interviews, documentation, and fault slippage data. Results: We identified challenges related to SoS with respect to fault slippage, test turn-around time, and test maintainability. We also classified the testing challenges to general testing challenges, challenges amplified by SoS, and challenges that are SoS specific. Interestingly, the interviewees agreed on the challenges, even though we sampled them with diversity in mind, which meant that the number of interviews conducted was sufficient to answer our research questions. We also identified solution proposals to the challenges that were categorized under four classes of developer quality assurance, function test, testing in all levels, and requirements engineering and communication. Conclusion: We conclude that although over half of the challenges we identified can be categorized as general testing challenges still SoS systems have their unique and amplified challenges stemming from SoS characteristics. Furthermore, it was found that interviews and fault slippage data indicated that different areas in the software process should be improved, which indicates that using only one of these methods would have led to an incomplete picture of the challenges in the case company.

  • 7.
    Ali, Nauman bin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Tanveer, Binish
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A Comparison of Citation Sources for Reference and Citation-Based Search in Systematic Literature Reviews2022In: e-Informatica Software Engineering Journal, ISSN 1897-7979, E-ISSN 2084-4840, Vol. 16, no 1, article id 220106Article, review/survey (Refereed)
    Abstract [en]

    Context: In software engineering, snowball sampling has been used as a supplementary and primary search strategy. The current guidelines recommend using Google Scholar (GS) for snowball sampling. However, the use of GS presents several challenges when using it as a source for citations and references. Objective: To compare the effectiveness and usefulness of two leading citation databases (GS and Scopus) for use in snowball sampling search. Method: We relied on a published study that has used snowball sampling as a search strategy and GS as the citation source. We used its primary studies to compute precision and recall for Scopus. Results: In this particular case, Scopus was highly effective with 95% recall and had better precision of 5.1% compared to GS’s 2.8%. Moreover, Scopus found nine additional relevant papers. On average, one would read approximately 15 extra papers in GS than Scopus to identify one additional relevant paper. Furthermore, Scopus supports batch downloading of both citations and papers’ references, has better quality metadata, and does better source filtering. Conclusion: This study suggests that Scopus seems to be more effective and useful for snowball sampling than GS for systematic secondary studies attempting to identify peer-reviewed literature. EVIE © 2022 The Authors.

    Download full text (pdf)
    fulltext
  • 8.
    Ali, Nauman bin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Usman, Muhammad
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A critical appraisal tool for systematic literature reviews in software engineering2019In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 112, p. 48-50Article, review/survey (Refereed)
    Abstract [en]

    Context: Methodological research on systematic literature reviews (SLRs)in Software Engineering (SE)has so far focused on developing and evaluating guidelines for conducting systematic reviews. However, the support for quality assessment of completed SLRs has not received the same level of attention. Objective: To raise awareness of the need for a critical appraisal tool (CAT)for assessing the quality of SLRs in SE. To initiate a community-based effort towards the development of such a tool. Method: We reviewed the literature on the quality assessment of SLRs to identify the frequently used CATs in SE and other fields. Results: We identified that the CATs currently used is SE were borrowed from medicine, but have not kept pace with substantial advancements in the field of medicine. Conclusion: In this paper, we have argued the need for a CAT for quality appraisal of SLRs in SE. We have also identified a tool that has the potential for application in SE. Furthermore, we have presented our approach for adapting this state-of-the-art CAT for assessing SLRs in SE. © 2019 The Authors

    Download full text (pdf)
    fulltext
  • 9.
    Ali, Nauman bin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Usman, Muhammad
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Reliability of search in systematic reviews: Towards a quality assessment framework for the automated-search strategy2018In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, ISSN 0950-5849, Vol. 99, p. 133-147Article in journal (Refereed)
    Abstract [en]

    Context: The trust in systematic literature reviews (SLRs) to provide credible recommendations is critical for establishing evidence-based software engineering (EBSE) practice. The reliability of SLR as a method is not a given and largely depends on the rigor of the attempt to identify, appraise and aggregate evidence. Previous research, by comparing SLRs on the same topic, has identified search as one of the reasons for discrepancies in the included primary studies. This affects the reliability of an SLR, as the papers identified and included in it are likely to influence its conclusions. Objective: We aim to propose a comprehensive evaluation checklist to assess the reliability of an automated-search strategy used in an SLR. Method: Using a literature review, we identified guidelines for designing and reporting automated-search as a primary search strategy. Using the aggregated design, reporting and evaluation guidelines, we formulated a comprehensive evaluation checklist. The value of this checklist was demonstrated by assessing the reliability of search in 27 recent SLRs. Results: Using the proposed evaluation checklist, several additional issues (not captured by the current evaluation checklist) related to the reliability of search in recent SLRs were identified. These issues severely limit the coverage of literature by the search and also the possibility to replicate it. Conclusion: Instead of solely relying on expensive replications to assess the reliability of SLRs, this work provides means to objectively assess the likely reliability of a search-strategy used in an SLR. It highlights the often-assumed aspect of repeatability of search when using automated-search. Furthermore, by explicitly considering repeatability and consistency as sub-characteristics of a reliable search, it provides a more comprehensive evaluation checklist than the ones currently used in EBSE. © 2018 Elsevier B.V.

  • 10.
    Ali, Nauman
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Evaluating strategies for study selection in systematic literature studies2014In: ESEM '14 Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ACM , 2014, Vol. article 45Conference paper (Refereed)
    Abstract [en]

    Context: The study selection process is critical to improve the reliability of secondary studies. Goal: To evaluate the selection strategies commonly employed in secondary studies in software engineering. Method: Building on these strate- gies, a study selection process was formulated and evalu- ated in a systematic review. Results: The selection process used a more inclusive strategy than the one typically used in secondary studies, which led to additional relevant articles. Conclusions: The results indicates that a good-enough sam- ple could be obtained by following a less inclusive but more efficient strategy, if the articles identified as relevant for the study are a representative sample of the population, and there is a homogeneity of results and quality of the articles.

    Download full text (pdf)
    fulltext
  • 11.
    Bennin, Kwabena Ebo
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Yu, Xiao
    City University of Hong Kong, HKG.
    Revisiting the Impact of Concept Drift on Just-in-Time Quality Assurance2020In: Proceedings - 2020 IEEE 20th International Conference on Software Quality, Reliability, and Security, QRS 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, p. 53-59, article id 9282807Conference paper (Refereed)
    Abstract [en]

    The performance of software defect prediction(SDP) models is known to be dependent on the datasets used for training the models. Evolving data in a dynamic software development environment such as significant refactoring and organizational changes introduces new concept to the prediction model, thus making improved classification performance difficult. In this study, we investigate and assess the existence and impact of concept drift on SDP performances. We empirically asses the prediction performance of five models by conducting cross-version experiments using fifty-five releases of five open-source projects. Prediction performance fluctuated as the training datasets changed over time. Our results indicate that the quality and the reliability of defect prediction models fluctuate over time and that this instability should be considered by software quality teams when using historical datasets. The performance of a static predictor constructed with data from historical versions may degrade over time due to the challenges posed by concept drift. © 2020 IEEE.

    Download full text (pdf)
    fulltext
  • 12.
    bin Ali, Nauman
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Operationalization of lean thinking through value stream mapping with simulation and FLOW2015Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Background: The continued success of Lean thinking beyond manufacturing has led to an increasing interest to utilize it in software engineering (SE). Value Stream Mapping (VSM) had a pivotal role in the operationalization of Lean thinking. However, this has not been recognized in SE adaptations of Lean. Furthermore, there are two main shortcomings in existing adaptations of VSM for an SE context. First, the assessments for the potential of the proposed improvements are based on idealistic assertions. Second, the current VSM notation and methodology are unable to capture the myriad of significant information flows, which in software development go beyond just the schedule information about the flow of a software artifact through a process. Objective: This thesis seeks to assess Software Process Simulation Modeling (SPSM) as a solution to the first shortcoming of VSM. In this regard, guidelines to perform simulation-based studies in industry are consolidated, and the usefulness of VSM supported with SPSM is evaluated. To overcome the second shortcoming of VSM, a suitable approach for capturing rich information flows in software development is identified and its usefulness to support VSM is evaluated. Overall, an attempt is made to supplement existing guidelines for conducting VSM to overcome its known shortcomings and support adoption of Lean thinking in SE. The usefulness and scalability of these proposals is evaluated in an industrial setting. Method: Three literature reviews, one systematic literature review, four industrial case studies, and a case study in an academic context were conducted as part of this research. Results: Little evidence to substantiate the claims of the usefulness of SPSM was found. Hence, prior to combining it with VSM, we consolidated the guidelines to conduct an SPSM based study and evaluated the use of SPSM in academic and industrial contexts. In education, it was found to be a useful complement to other teaching methods, and in the industry, it triggered useful discussions and was used to challenge practitioners’ perceptions about the impact of existing challenges and proposed improvements. The combination of VSM with FLOW (a method and notation to capture information flows, since existing VSM adaptions for SE are insufficient for this purpose) was successful in identifying challenges and improvements related to information needs in the process. Both proposals to support VSM with simulation and FLOW led to identification of waste and improvements (which would not have been possible with conventional VSM), generated more insightful discussions and resulted in more realistic improvements. Conclusion: This thesis characterizes the context and shows how SPSM was beneficial both in the industrial and academic context. FLOW was found to be a scalable, lightweight supplement to strengthen the information flow analysis in VSM. Through successful industrial application and uptake, this thesis provides evidence of the usefulness of the proposed improvements to the VSM activities.

    Download full text (pdf)
    FULLTEXT01
  • 13.
    bin Ali, Nauman
    Blekinge Institute of Technology, School of Computing.
    Towards Guidelines for Conducting Software Process Simulation in Industry2013Licentiate thesis, comprehensive summary (Other academic)
    Abstract [en]

    Background: Since the 1950s explicit software process models have been used for planning, executing and controlling software development activities. To overcome the limitation of static models at capturing the inherent dynamism in software development, Software Process Simulation Modelling (SPSM) was introduced in the late 1970s. SPSM has been used to address various challenges, e.g. estimation, planning and process assessment. The simulation models developed over the years have varied in their scope, purpose, approach and the application domain. However, there is a need to aggregate the evidence regarding the usefulness of SPSM for achieving its intended purposes. Objective: This thesis aims to facilitate adoption of SPSM in industrial practice by exploring two directions. Firstly it aims to establish the usefulness of SPSM for its intended purposes, e.g. for planning, training and as an alternative to study the real world software (industrial and open source) development. Secondly to define and evaluate a process for conducting SPSM studies in industry. Method: Two systematic literature reviews (SLR), a literature review, a case study and an action research study were conducted. A literature review of existing SLRs was done to identify the strategies for selecting studies. The resulting process for study selection was utilized in an SLR to capture and aggregate evidence regarding the usefulness of SPSM. Another SLR was used to identify existing process descriptions of how to conduct an SPSM study. The consolidated process and associated guidelines identified in this review were used in an action research study to develop a simulation model of the testing process in a large telecommunication vendor. The action research was preceded by a case study to understand the testing process at the company. Results: A study selection process based on the strategies identified from literature was proposed. It was found to systemize selection and to support inclusiveness with reasonable additional effort in an SLR of the SPSM literature. The SPSM studies identified in literature scored poorly on the rigor and relevance criteria and lacked evaluation of SPSM for the intended purposes. Lastly, based on literature, a six-step process to conduct an SPSM study was used to develop a System Dynamics model of the testing process for training purposes in the company. Conclusion: The findings identify two potential directions for facilitating SPSM adoption. First, by learning from other disciplines having done simulation for a longer time. It was evident how similar the consolidated process for conducting an SPSM study was to the process used in simulation in general. Second the existing work on SPSM can at best be classified as strong ``proof-of-concept’’ that SPSM can be useful in the real world software development. Thus, there is a need to evaluate and report the usefulness of SPSM for the intended purposes with scientific rigor.

    Download full text (pdf)
    FULLTEXT01
  • 14.
    Bin Ali, Nauman
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Blekinge Inst Technol, Karlskrona, Sweden..
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. Blekinge Inst Technol, Karlskrona, Sweden..
    Nicolau de Franca, Breno Bernard
    Univ Fed Rio de Janeiro, ESE Grp, PESC COPPE, BR-68511 Rio De Janeiro, Brazil..
    Evaluation of simulation-assisted value stream mapping for software product development: Two industrial cases2015In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 68, p. 45-61Article in journal (Refereed)
    Abstract [en]

    Context: Value stream mapping (VSM) as a tool for lean development has led to significant improvements in different industries. In a few studies, it has been successfully applied in a software engineering context. However, some shortcomings have been observed in particular failing to capture the dynamic nature of the software process to evaluate improvements i.e. such improvements and target values are based on idealistic situations. Objective: To overcome the shortcomings of VSM by combining it with software process simulation modeling, and to provide reflections on the process of conducting VSM with simulation. Method: Using case study research, VSM was used for two products at Ericsson AB, Sweden. Ten workshops were conducted in this regard. Simulation in this study was used as a tool to support discussions instead of as a prediction tool. The results have been evaluated from the perspective of the participating practitioners, an external observer, and reflections of the researchers conducting the simulation that was elicited by the external observer. Results: Significant constraints hindering the product development from reaching the stated improvement goals for shorter lead time were identified. The use of simulation was particularly helpful in having more insightful discussions and to challenge assumptions about the likely impact of improvements. However, simulation results alone were found insufficient to emphasize the importance of reducing waiting times and variations in the process. Conclusion: The framework to assist VSM with simulation presented in this study was successfully applied in two cases. The involvement of various stakeholders, consensus building steps, emphasis on flow (through waiting time and variance analysis) and the use of simulation proposed in the framework led to realistic improvements with a high likelihood of implementation. (C) 2015 Elsevier B.V. All rights reserved.

  • 15.
    bin Ali, Nauman
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Wohlin, Claes
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A systematic literature review on the industrial use of software process simulation2014In: Journal of Systems and Software, ISSN 0164-1212, Vol. 97Article in journal (Refereed)
    Abstract [en]

    Context Software process simulation modelling (SPSM) captures the dynamic behaviour and uncertainty in the software process. Existing literature has conflicting claims about its practical usefulness: SPSM is useful and has an industrial impact; SPSM is useful and has no industrial impact yet; SPSM is not useful and has little potential for industry. Objective To assess the conflicting standpoints on the usefulness of SPSM. Method A systematic literature review was performed to identify, assess and aggregate empirical evidence on the usefulness of SPSM. Results In the primary studies, to date, the persistent trend is that of proof-of-concept applications of software process simulation for various purposes (e.g. estimation, training, process improvement, etc.). They score poorly on the stated quality criteria. Also only a few studies report some initial evaluation of the simulation models for the intended purposes. Conclusion There is a lack of conclusive evidence to substantiate the claimed usefulness of SPSM for any of the intended purposes. A few studies that report the cost of applying simulation do not support the claim that it is an inexpensive method. Furthermore, there is a paramount need for improvement in conducting and reporting simulation studies with an emphasis on evaluation against the intended purpose.

    Download full text (pdf)
    fulltext
  • 16.
    bin Ali, Nauman
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Unterkalmsteiner, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Use and evaluation of simulation for software process education: a case study2014Conference paper (Refereed)
    Abstract [en]

    Software Engineering is an applied discipline and concepts are difficult to grasp only at a theoretical level alone. In the context of a project management course, we introduced and evaluated the use of software process simulation (SPS) based games for improving students’ understanding of software development processes. The effects of the intervention were measured by evaluating the students’ arguments for choosing a particular development process. The arguments were assessed with the Evidence-Based Reasoning framework, which was extended to assess the strength of an argument. The results indicate that students generally have difficulty providing strong arguments for their choice of process models. Nevertheless, the assessment indicates that the intervention of the SPS game had a positive impact on the students’ arguments. Even though the illustrated argument assessment approach can be used to provide formative feedback to students, its use is rather costly and cannot be considered a replacement for traditional assessments.

    Download full text (pdf)
    fulltext
  • 17.
    Bjarnason, Elizabeth
    et al.
    Lund University.
    Åberg, Patrik
    Ericsson AB.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Software selection in large-scale software engineering: A model and criteria based on interactive rapid reviews2023In: Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 28, no 2, article id 51Article in journal (Refereed)
    Abstract [en]

    Context: Software selection in large-scale software development continues to be ad hoc and ill-structured. Previous proposals for software component selection tend to be technology-specific and/or do not consider business or ecosystem concerns. Objective: Our main aim is to develop an industrially relevant technology-agnostic method that can support practitioners in making informed decisions when selecting software components for use in tools or in products based on a holistic perspective of the overall environment. Method: We used method engineering to iteratively develop a software selection method for Ericsson AB based on a combination of published research and practitioner insights. We used interactive rapid reviews to systematically identify and analyse scientific literature and to support close cooperation and co-design with practitioners from Ericsson. The model has been validated through a focus group and by practical use at the case company. Results: The model consists of a high-level selection process and a wide range of criteria for assessing and for evaluating software to include in business products and tools. Conclusions: We have developed an industrially relevant model for component selection through active engagement from a company. Co-designing the model based on previous knowledge demonstrates a viable approach to industry-academia collaboration and provides a practical solution that can support practitioners in making informed decisions based on a holistic analysis of business, organisation and technical factors. © 2023, The Author(s).

    Download full text (pdf)
    fulltext
  • 18.
    Börstler, Jürgen
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. University of Applied Sciences, Germany.
    Double-counting in software engineering tertiary studies — An overlooked threat to validity2023In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 158, article id 107174Article, review/survey (Refereed)
    Abstract [en]

    Context: Double-counting in a literature review occurs when the same data, population, or evidence is erroneously counted multiple times during synthesis. Detecting and mitigating the threat of double-counting is particularly challenging in tertiary studies. Although this topic has received much attention in the health sciences, it seems to have been overlooked in software engineering. Objective: We describe issues with double-counting in tertiary studies, investigate the prevalence of the issue in software engineering, and propose ways to identify and address the issue. Method: We analyze 47 tertiary studies in software engineering to investigate in which ways they address double-counting and whether double-counting might be a threat to validity in them. Results: In 19 of the 47 tertiary studies, double-counting might bias their results. Of those 19 tertiary studies, only 5 consider double-counting a threat to their validity, and 7 suggest strategies to address the issue. Overall, only 9 of the 47 tertiary studies, acknowledge double-counting as a potential general threat to validity for tertiary studies. Conclusions: Double-counting is an overlooked issue in tertiary studies in software engineering, and existing design and evaluation guidelines do not address it sufficiently. Therefore, we propose recommendations that may help to identify and mitigate double-counting in tertiary studies. © 2023 The Author(s)

    Download full text (pdf)
    fulltext
  • 19.
    Börstler, Jürgen
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Svensson, Martin
    Blekinge Institute of Technology, Faculty of Engineering, Department of Industrial Economics.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering. University of Applied Sciences Flensburg, Germany.
    Investigating Acceptance Behavior in Software Engineering – Theoretical Perspectives2023In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 198, article id 111592Article in journal (Refereed)
    Abstract [en]

    Background: Software engineering research aims to establish software development practice on a scientific basis. However, the evidence of the efficacy of technology is insufficient to ensure its uptake in industry. In the absence of a theoretical frame of reference, we mainly rely on best practices and expert judgment from industry-academia collaboration and software process improvement research to improve the acceptance of the proposed technology. Objective: To identify acceptance models and theories and discuss their applicability in the research of acceptance behavior related to software development.Method: We analyzed literature reviews within an interdisciplinary team to identify models and theories relevant to software engineering research. We further discuss acceptance behavior from the human information processing perspective of automatic and affect-driven processes (“fast” system 1 thinking) and rational and rule-governed processes (“slow” system 2 thinking). Results: We identified 30 potentially relevant models and theories. Several of them have been used in researching acceptance behavior in contexts related to software development, but few have been validated in such contexts. They use constructs that capture aspects of (automatic) system 1 and (rational) system 2 oriented processes. However, their operationalizations focus on system 2-oriented processes indicating a rational view of behavior, thus overlooking important psychological processes underpinning behavior. Conclusions: Software engineering research may use acceptance behavior models and theories more extensively to understand and predict practice adoption in the industry. Such theoretical foundations will help improve the impact of software engineering research. However, more consideration should be given to their validation, overlap, construct operationalization, and employed data collection mechanisms when using these models and theories.

    Download full text (pdf)
    fulltext
  • 20.
    de Franca, Breno Bernard Nicolau
    et al.
    Universidade Estadual de Campinas, BRA.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    The Role of Simulation-Based Studies in Software Engineering Research2020In: Contemporary Empirical Methods in Software Engineering / [ed] Michael Felderer, Guilherme Horta Travassos, Springer Nature, 2020, p. 263-287Chapter in book (Refereed)
    Abstract [en]

    Several decades ago, inspired by other knowledge areas, simulation was introduced as a research method to Software Engineering. Motivated by potential benefits achieved in other areas, the software engineering community has used simulation-based studies for planning, controlling, and improving software development. However, unclear expectations from simulation-based studies, a lack of methodological support, as well as dispersed knowledge to support model building and calibration have hindered widespread adoption of simulation-based investigations. In this chapter, we delineate the role of simulation in software engineering research and compile processes and guidelines into a comprehensive life cycle. This chapter aims to guide software engineering researchers to conduct effective simulation-based studies in real-world settings.

  • 21. Edison, Henry
    et al.
    bin Ali, Nauman
    Blekinge Institute of Technology, School of Computing.
    Torkar, Richard
    Blekinge Institute of Technology, School of Computing.
    Towards innovation measurement in the software industry2013In: Journal of Systems and Software, ISSN 0164-1212, Vol. 86, no 5, p. 1390-1407Article in journal (Refereed)
    Abstract [en]

    In today's highly competitive business environments with shortened product and technology life cycle, it is critical for software industry to continuously innovate. This goal can be achieved by developing a better understanding and control of the activities and determinants of innovation. Innovation measurement initiatives assess innovation capability, output and performance to help develop such an understanding. This study explores various aspects relevant to innovation measurement ranging from definitions, measurement frameworks and metrics that have been proposed in literature and used in practice. A systematic literature review followed by an online questionnaire and interviews with practitioners and academics were employed to identify a comprehensive definition of innovation that can be used in software industry. The metrics for the evaluation of determinants, inputs, outputs and performance were also aggregated and categorised. Based on these findings, a conceptual model of the key measurable elements of innovation was constructed from the findings of the systematic review. The model was further refined after feedback from academia and industry through interviews.

  • 22.
    Engström, Emelie
    et al.
    Lund University, SWE.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Bjarnason, Elizabeth
    Lund University, SWE.
    SERP-test: a taxonomy for supporting industry-academia communication2017In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 25, no 4, p. 1269-1305Article in journal (Refereed)
    Abstract [en]

    This paper presents the construction and evaluation of SERP-test, a taxonomy aimed to improve communication between researchers and practitioners in the area of software testing. SERP-test can be utilized for direct communication in industry academia collaborations. It may also facilitate indirect communication between practitioners adopting software engineering research and researchers who are striving for industry relevance. SERP-test was constructed through a systematic and goal-oriented approach which included literature reviews and interviews with practitioners and researchers. SERP-test was evaluated through an online survey and by utilizing it in an industry–academia collaboration project. SERP-test comprises four facets along which both research contributions and practical challenges may be classified: Intervention, Scope, Effect target and Context constraints. This paper explains the available categories for each of these facets (i.e., their definitions and rationales) and presents examples of categorized entities. Several tasks may benefit from SERP-test, such as formulating research goals from a problem perspective, describing practical challenges in a researchable fashion, analyzing primary studies in a literature review, or identifying relevant points of comparison and generalization of research.

  • 23.
    Iftikhar, Umar
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Usman, Muhammad
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A catalog of source code metrics – a tertiary study2023In: Software Quality: Higher Software Quality through Zero Waste Development / [ed] Daniel Mendez, Dietmar Winkler, Johannes Kross, Stefan Biffl, Johannes Bergsmann, Springer, 2023, Vol. 472, p. 87-106Conference paper (Refereed)
    Abstract [en]

    Context: A large number of source code metrics are reported in the literature. It is necessary to systematically collect, describe and classify source code metrics to support research and practice.Objective: We aim to utilize existing secondary studies to develop a cat- alog of source code metrics together with their descriptions. The catalog will also provide information about which units of code (e.g., operators, operands, lines of code, variables, parameters, code blocks, or functions) are used to measure the internal quality attributes and the scope on which they are collected. 

    Method: We conducted a tertiary study to identify secondary studies re- porting source code metrics. We have classified the source code metrics according to the measured internal quality attributes, the units of code used in the measures, and the scope at which the source code metrics are collected. 

    Results: From 711 secondary studies, we identified 52 relevant secondary studies. We reported 423 source code metrics together with their de- scriptions and the internal quality attributes they measure. Source code metrics predominantly incorporate function as a unit of code to measure internal quality attributes. In contrast, several source code metrics use more than one unit of code when measuring internal quality attributes. Nearly 51% of the source code metrics are collected at the class scope, while almost 12% and 15% of source code metrics are collected at module and application levels, respectively. 

    Conclusions: Researchers and practitioners can use the extensive catalog to assess which source code metrics meet their individual needs based on the description and classification scheme presented. 

  • 24.
    Iftikhar, Umar
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Usman, Muhammad
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A tertiary study on links between source code metrics and external quality attributes2024In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 165, article id 107348Article, review/survey (Refereed)
    Abstract [en]

    Context: Several secondary studies have investigated the relationship between internal quality attributes, source code metrics and external quality attributes. Sometimes they have contradictory results. Objective: We synthesize evidence of the link between internal quality attributes, source code metrics and external quality attributes along with the efficacy of the prediction models used. Method: We conducted a tertiary review to identify, evaluate and synthesize secondary studies. We used several characteristics of secondary studies as indicators for the strength of evidence and considered them when synthesizing the results. Results: From 711 secondary studies, we identified 15 secondary studies that have investigated the link between source code and external quality. Our results show : (1) primarily, the focus has been on object-oriented systems, (2) maintainability and reliability are most often linked to internal quality attributes and source code metrics, with only one secondary study reporting evidence for security, (3) only a small set of complexity, coupling, and size-related source code metrics report a consistent positive link with maintainability and reliability, and (4) group method of data handling (GMDH) based prediction models have performed better than other prediction models for maintainability prediction. Conclusions: Based on our results, lines of code, coupling, complexity and the cohesion metrics from Chidamber & Kemerer (CK) metrics are good indicators of maintainability with consistent evidence from high and moderate-quality secondary studies. Similarly, four CK metrics related to coupling, complexity and cohesion are good indicators of reliability, while inheritance and certain cohesion metrics show no consistent evidence of links to maintainability and reliability. Further empirical studies are needed to explore the link between internal quality attributes, source code metrics and other external quality attributes, including functionality, portability, and usability. The results will help researchers and practitioners understand the body of knowledge on the subject and identify future research directions. © 2023 The Author(s)

    Download full text (pdf)
    fulltext
  • 25.
    Iftikhar, Umar
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    On potential improvements in the analysis of the evolution of themes in code review comments2023In: Proceedings - 2023 49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 340-347Conference paper (Refereed)
    Abstract [en]

    Context: The modern code review process is considered an essential quality assurance step in software development. The code review comments generated can provide insights regarding source code quality and development practices. However, the large number of code review comments makes it challenging to identify interesting patterns manually. In a recent study, Wen et al. used traditional topic modeling to analyze the evolution of code review comments. Their approach could identify interesting patterns that may lead to improved development practices.Objective: In this study, we investigate potential improvements to Wen et al.'s state-of-the-art approach to analyze the evolution of code review comments.Method: We used 209,166 code review comments from three open-source systems to explore and empirically analyze alternative design and implementation choices and demonstrate their impact.Results: We identified the following potential improvements to the current state-of-the-art as described by Wen et al.: 1) utilize a topic modeling method that is optimized for short texts, 2) a refined approach for identifying a suitable number of topics, and 3) a more elaborate approach for analyzing topic evolution. Our results indicate that the proposed changes have quantitatively different results than the current approach. The qualitative interpretation of the topics generated with our changes indicates their usefulness.Conclusions: Our results indicate the potential usefulness of changes to state-of-the-art approaches to analyzing the evolution of code review comments, with practical implications for researchers and practitioners. However, further research is required to compare the effectiveness of both approaches. © 2023 IEEE.

    Download full text (pdf)
    fulltext
  • 26.
    Iftikhar, Umar
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Kopp, Oliver
    University of Stuttgart, Germany.
    Identifying prevalent quality issues in code changes by analyzing reviewers' feedback2024Manuscript (preprint) (Other academic)
    Abstract [en]

    Context: Code reviewers provide valuable feedback during the code review. Identifying common issues described in the reviewers' feedback can provide input for context-specific software improvement opportunities. However, the use of reviewer feedback for this purpose is currently less explored.

    Objective: Assessing if and how automation can derive themes in reviewers' feedback and whether these themes help to identify recurring quality-related issues in code changes.

    Method: We conducted a case study using the JabRef system to distinguish reviewers' feedback on merged and abandoned code changes for the analysis. We used topic modeling to identify themes in 5,560 code review comments. The resulting themes were analyzed and named by a domain expert from JabRef.

    Results: The domain expert considered the identified themes from the proposed automation approach to represent quality-related issues. We found that different quality issues are pointed out in code reviews for merged and abandoned code changes. 

    Conclusions: The results indicate the usefulness of our proposed automation approach in utilizing code review comments for understanding the prevalent code quality issues that can help derive targeted and context-bound improvement actions.

  • 27.
    Jabbari, Ramtin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Tanveer, Binish
    Fraunhofer Institute for Experimental Software Engineering IESE, DEU.
    Towards a benefits dependency network for DevOps based on a systematic literature review2018In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 30, no 11, article id e1957Article in journal (Refereed)
    Abstract [en]

    DevOps as a new way of thinking for software development and operations has received much attention in the industry, while it has not been thoroughly investigated in academia yet. The objective of this study is to characterize DevOps by exploring its central components in terms of principles, practices and their relations to the principles, challenges of DevOps adoption, and benefits reported in the peer-reviewed literature. As a key objective, we also aim to realize the relations between DevOps practices and benefits in a systematic manner. A systematic literature review was conducted. Also, we used the concept of benefits dependency network to synthesize the findings, in particular, to specify dependencies between DevOps practices and link the practices to benefits. We found that in many cases, DevOps characteristics, ie, principles, practices, benefits, and challenges, were not sufficiently defined in detail in the peer-reviewed literature. In addition, only a few empirical studies are available, which can be attributed to the nascency of DevOps research. Also, an initial version of the DevOps benefits dependency network has been derived. The definition of DevOps principles and practices should be emphasized given the novelty of the concept. Further empirical studies are needed to improve the benefits dependency network presented in this study. © 2018 John Wiley & Sons, Ltd.

  • 28.
    Jabbari, Ramtin
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman Bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Tanveer, Binish
    What is DevOps?: A Systematic Mapping Study on Definitions and Practices2016Conference paper (Refereed)
  • 29.
    Josyula, Jitendra
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Panamgipalli, Sarat
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Usman, Muhammad
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Britto, Ricardo
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Software Practitioners' Information Needs and Sources: A Survey Study2018In: Proceedings - 2018 9th International Workshop on Empirical Software Engineering in Practice, IWESEP 2018, IEEE , 2018, p. 1-6Conference paper (Refereed)
    Abstract [en]

    Software engineering practitioners have information needs to support strategic, tactical and operational decision-making. However, there is scarce research on understanding which information needs exist and how they are currently fulfilled in practice. This study aims to identify the information needs, the frequency of their occurrence, the sources of information used to satisfy the needs, and the perception of practitioners regarding the usefulness of the sources currently used. For this purpose, a literature review was conducted to aggregate the current state of understanding in this area. We built on the results of the literature review and developed further insights through in-depth interviews with 17 practitioners. We further triangulated the findings from these two investigations by conducting a web-based survey (with 83 completed responses). Based on the results, we infer that information regarding product design, product architecture and requirements gathering are the most frequently faced needs. Software practitioners mostly use blogs, community forums, product documentation, and discussion with colleagues to address their information needs.

  • 30.
    Laiq, Muhammad
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Engström, Emelie
    Lund University.
    A data-driven approach for understanding invalid bug reports: An industrial case study2023In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 164, article id 107305Article in journal (Refereed)
    Abstract [en]

    Context: Bug reports created during software development and maintenance do not always describe deviations from a system's valid behavior. Such invalid bug reports may consume significant resources and adversely affect the prioritization and resolution of valid bug reports. There is a need to identify preventive actions to reduce the inflow of invalid bug reports. Existing research has shown that manually analyzing invalid bug report descriptions provides cues regarding preventive actions. However, such a manual approach is not cost-effective due to the time required to analyze a sufficiently large number of bug reports needed to identify useful patterns. Furthermore, the analysis needs to be repeated as the underlying causes of invalid bug reports change over time. Objective: In this study, we propose and evaluate the use of Latent Dirichlet Allocation (LDA), a topic modeling approach, to support practitioners in suggesting preventive actions to avoid the creation of similar invalid bug reports in the future. Method: In an industrial case study, we first manually analyzed descriptions of invalid bug reports to identify common patterns in their descriptions. We further investigated to what extent LDA can support this manual process. We used expert-based validation to evaluate the relevance of identified common patterns and their usefulness in suggesting preventive measures. Results: We found that invalid bug reports have common patterns that are perceived as relevant, and they can be used to devise preventive measures. Furthermore, the identification of common patterns can be supported with automation. Conclusion: Using LDA, practitioners can effectively identify representative groups of bug reports (i.e., relevant common patterns) from a large number of bug reports and analyze them further to devise preventive measures. © 2023 The Author(s)

    Download full text (pdf)
    fulltext
  • 31.
    Laiq, Muhammad
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Engström, Emelie
    Lund University.
    Early Identification of Invalid Bug Reports in Industrial Settings – A Case Study2022In: Product-Focused Software Process Improvement / [ed] Taibi D., Kuhrmann M., Mikkonen T., Abrahamsson P., Klünder J., Springer Science+Business Media B.V., 2022, p. 497-507Conference paper (Refereed)
    Abstract [en]

    Software development companies spend considerable time resolving bug reports. However, bug reports might be invalid, i.e., not point to a valid flaw. Expensive resources and time might be expended on invalid bug reports before discovering that they are invalid. In this case study, we explore the impact of invalid bug reports and develop and assess the use of machine learning (ML) to indicate whether a bug report is likely invalid. We found that about 15% of bug reports at the case company are invalid, and that their resolution time is similar to valid bug reports. Among the ML-based techniques we used, logistic regression and SVM show promising results. In the feedback, practitioners indicated an interest in using the tool to identify invalid bug reports at early stages. However, they emphasized the need to improve the explainability of ML-based recommendations and to reduce the maintenance cost of the tool. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

  • 32.
    Mendes, Emilia
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Counsell, Steve
    Brunel University London, GBR.
    Baldassare, Maria Teresa
    Università degli Studi di Bari, ITA.
    Special issue on evaluation and assessment in software engineering2019In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 151, p. 224-225Article in journal (Refereed)
  • 33.
    Minhas, Nasir Mehmood
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Wnuk, Krzysztof
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Regression testing goals: View of practitioners and researchers2017In: 24th Asia-Pacific Software Engineering Conference Workshops (APSECW), IEEE, 2017, p. 25-32Conference paper (Refereed)
    Abstract [en]

    Context: Regression testing is a well-researched area. However, the majority regression testing techniques proposed by the researchers are not getting the attention of the practitioners. Communication gaps between industry and academia and disparity in the regression testing goals are the main reasons. Close collaboration can help in bridging the communication gaps and resolving the disparities.Objective: The study aims at exploring the views of academics and practitioners about the goals of regression testing. The purpose is to investigate the commonalities and differences in their viewpoints and defining some common goals for the success of regression testing.Method: We conducted a focus group study, with 7 testing experts from industry and academia. 4 testing practitioners from 2companies and 3 researchers from 2 universities participated in the study. We followed GQM approach, to elicit the regression testing goals, information needs, and measures.Results: 43 regression testing goals were identified by the participants, which were reduced to 10 on the basis of similarity among the identified goals. Later during the priority assignment process, 5 goals were discarded, because the priority assigned to these goals was very low. Participants identified 47 information needs/questions required to evaluate the success of regression testing with reference to goal G5 (confidence). Which were then reduced to10 on the basis of similarity. Finally, we identified measures to gauge those information needs/questions, which were corresponding to the goal (G5).Conclusions: We observed that participation level of practitioners and researchers during the elicitation of goals and questions was same. We found a certain level of agreement between the participants regarding the regression testing definitions and goals.But there was some level of disagreement regarding the priorities of the goals. We also identified the need to implement a regression testing evaluation framework in the participating companies.

    Download full text (pdf)
    fulltext
  • 34.
    Molléri, Jefferson Seide
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Petersen, Kai
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Minhas, Tahir Nawaz
    Chatzipetrou, Panagiota
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Teaching students critical appraisal of scientific literature using checklists2018In: PROCEEDINGS OF THE 3RD EUROPEAN CONFERENCE OF SOFTWARE ENGINEERING EDUCATION (ECSEE), Association for Computing Machinery , 2018, p. 8-17Conference paper (Refereed)
    Abstract [en]

    Background: Teaching students to critically appraise scientific literature is an important goal for a postgraduate research methods course. Objective: To investigate the application of checklists for assessing the scientific rigor of empirical studies support students in reviewing case study research and experiments. Methods:We employed an experimental design where 76 students (in pairs) used two checklists to evaluate two papers (reporting a case study and an experiment) each. We compared the students' assessments against ratings from more senior researchers. We also collected data on students' perception of using the checklists. Results: The consistency of students' ratings and the accuracy when compared to ratings from seniors varied. A factor seemed to be that the clearer the reporting, the easier it is for students to judge the quality of studies. Students perceived checklist items related to data analysis as difficult to assess. Conclusion: As expected, this study reinforces the needs for clear reporting, as it is important that authors write to enable synthesis and quality assessment. With clearer reporting, the novices performed well in assessing the quality of the empirical work, which supports its continued use in the course as means for introducing scientific reviews. © 2018 Association for Computing Machinery.

  • 35.
    Petersen, Kai
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    An analysis of top author citations in software engineering and a comparison with other fields2021In: Scientometrics, ISSN 0138-9130, E-ISSN 1588-2861, Vol. 126, no 11, p. 9147-9183Article in journal (Refereed)
    Abstract [en]

    Ioannidis et al. provided a science-wide database of author citations. The data offers an opportunity to researchers in a field to compare the citation behavior of their field with others. In this paper, we conduct a systematic analysis of citations describing the situation in software engineering and compare it with the fields included in the data provided by Ioannidis et al. For comparison, we take the measures used by Ioannidis into consideration. We also report the top-scientists and investigate software engineering researchers’ activities in other fields. The data was obtained and provided by Ioannidis et al. based on the Scopus database. Our method for analysis focuses on descriptive statistics. We compared software engineering with other fields and reported demographic information for the top authors. The analysis was done without any modifications to the ranking. In the later analysis, we observed that 37% of researchers listed as software engineers were not in the software engineering field. On the other hand, the database included a large portion of top authors (ca. 60% to 80%) identified in other software engineering rankings. Other fields using the database are advised to review the author lists for their fields. Our research’s main risk was that researchers are listed that do not belong to our studied field. © 2021, The Author(s).

    Download full text (pdf)
    fulltext
  • 36.
    Petersen, Kai
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Nauman Bin, Ali
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Operationalizing the requirements selection process with study selection procedures from systematic literature reviews2015In: CEUR Workshop Proceedings, 2015, Vol. 1342, p. 102-113Conference paper (Refereed)
    Abstract [en]

    Context: Software organizations working in a market-driven environment have to select requirements from a large pool to be prioritized and put into backlogs for the development organization. Objective: This paper proposes an approach based on study selection in systematic literature reviews and translates the concept to requirements engineering. The rational for doing so is that the selection processes used there have been e?ective (selecting and finding relevant papers) and efficient (possible to use for a high number of studies, in some cases 10,000 research contributions had to be evaluated). Method: This paper can be classified as a solution proposal, and utilizes hypothetical examples to explain and argue for the method design decisions. Results: The process proposed consists of three main phases, namely establish selection criteria, evaluate selection criteria, and apply selection. On a more fine-grained level, nine activities are specified. Conclusion: Given that the process has been e?ective and efficient in a similar context, our proposition to be evaluated in future research contributions is that the process leads to e?ective and efficient decision making in requirements selection. © 2015 by the authors.

  • 37.
    Rico, Sergio
    et al.
    Lund University.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Engström, Emelie
    Lund University.
    Höst, Martin
    Lund University.
    Experiences from conducting rapid reviews in collaboration with practitioners — Two industrial cases2024In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 167, article id 107364Article in journal (Refereed)
    Abstract [en]

    Context: Evidence-based software engineering (EBSE) aims to improve research utilization in practice. It relies on systematic methods to identify, appraise, and synthesize existing research findings to answer questions of interest for practice. However, the lack of practitioners’ involvement in these studies’ design, execution, and reporting indicates a lack of appreciation for the need for knowledge exchange between researchers and practitioners. The resultant systematic literature studies often lack relevance for practice. Objective: This paper explores the use of Rapid Reviews (RRs), in fostering knowledge exchange between academia and industry. Through the lens of two case studies, we delve into the practical application and experience of conducting RRs. Methods: We analyzed the conduct of two rapid reviews by two different groups of researchers and practitioners. We collected data through interviews, and the documents produced during the review (like review protocols, search results, and presentations). The interviews were analyzed using thematic analysis. Results: We report how the two groups of researchers and practitioners performed the rapid reviews. We observed some benefits, like promoting dialogue and paving the way for future collaborations. We also found that practitioners entrusted the researchers to develop and follow a rigorous approach and were more interested in the applicability of the findings in their context. The problems investigated in these two cases were relevant but not the most immediate ones. Therefore, rapidness was not a priority for the practitioners. Conclusion: The study illustrates that rapid reviews can support researcher-practitioner communication and industry-academia collaboration. Furthermore, the recommendations based on the experiences from the two cases complement the detailed guidelines researchers and practitioners may follow to increase interaction and knowledge exchange. © 2023 The Author(s)

    Download full text (pdf)
    fulltext
  • 38.
    Singh, Shailesh Pratap
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Lundberg, Lars
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    Smart and Adaptive Architecture for a Dedicated Internet of Things Network Comprised of Diverse Entities: A Proposal and Evaluation2022In: Sensors, E-ISSN 1424-8220, Vol. 22, no 8, article id 3017Article in journal (Refereed)
    Abstract [en]

    Advances in 5G and the Internet of Things (IoT) have to cater to the diverse and varying needs of different stakeholders, devices, sensors, applications, networks, and access technologies that come together for a dedicated IoT network for a synergistic purpose. Therefore, there is a need for a solution that can assimilate the various requirements and policies to dynamically and intelligently orchestrate them in the dedicated IoT network. Thus we identify and describe a representative industry-relevant use case for such a smart and adaptive environment through interviews with experts from a leading telecommunication vendor. We further propose and evaluate candidate architectures to achieve dynamic and intelligent orchestration in such a smart environment using a systematic approach for architecture design and by engaging six senior domain and IoT experts. The candidate architecture with an adaptive and intelligent element (“Smart AAA agent”) was found superior for modifiability, scalability, and performance in the assessments. This architecture also explores the enhanced role of authentication, authorization, and accounting (AAA) and makes the base for complete orchestration. The results indicate that the proposed architecture can meet the requirements for a dedicated IoT network, which may be used in further research or as a reference for industry solutions. © 2022 by the authors. Licensee MDPI, Basel, Switzerland.

    Download full text (pdf)
    fulltext
  • 39.
    Singh, Shailesh Pratap
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    Lundberg, Lars
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Performance Trade-offs of Two Software Architectures Executing Over Cloud and Fog for an IoT NetworkManuscript (preprint) (Other academic)
  • 40.
    Singh, Shailesh Pratap
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    Lundberg, Lars
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Sidorova, Yulia
    Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
    From Traditional to Next Generation AAA: A reference Architecture for a dedicated IoT Network2019In: Proceedings of the IV International Scientific Conference “Convergent Cognitive Information Technologies”, 2019Conference paper (Refereed)
    Abstract [en]

    Emerging scenarios for the “Internet of Things” (IoT) require a dedicated software defined network over the conventional communication network provided by the different service providers and free to use communication methodologies. These IoT networks have their own dedicated requirements, based on the different stakeholders involved in it, which can be realized via dynamic context-based authentication, authorization and accounting (AAA). This AAA needs to be envisaged in a much larger perspective than the current perspective in telecom networks. As part of this study, we have identified a few external stakeholders, who are domain and IoT experts and discussed the various requirements, scenarios and change scenarios for the dedicated IoT networks. Relying on Zachman’s framework, a reference architecture that we call as “Smart AAA agent for dedicated IoT network” is presented to the domain experts and evaluated against their scenarios utilizing a scenario-based software architecture analysis method. The scenarios discussed and utilized for the analysis encompass two ends of the IoT spectrum of requirements. The medical domain scenarios have critical IoT perspective as lives and health of patients is involved, while the enterprise IoT scenarios involve huge scalability and monetizability aspect, which is very important for the industry. With this reference architecture, we demonstrate a system capable of providing a software defined network fulfilling the requirements of a dedicated IoT network as enlisted in scenarios by the external stakeholders. Furthermore, this proposed reference architecture is evaluated with a software architect and matured to its current state and made available for any future research, development or standardization for 5G and next generation networks for the Internet of Things.

  • 41.
    Tanveer, Binish
    et al.
    Fraunhofer Institute for Experimental Software Engineering IESE,, DEU.
    Vollmer, Anna Maria
    Fraunhofer Institute for Experimental Software Engineering IESE,, DEU.
    Braun, Stefan
    Insiders Technologies GmBH, DEU.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    An evaluation of effort estimation supported by change impact analysis in agile software development2019In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 31, no 5, article id e2165Article in journal (Refereed)
    Abstract [en]

    In agile software development, functionality is added to the system in an incremental and iterative manner. Practitioners often rely on expert judgment to estimate the effort in this context. However, the impact of a change on the existing system can provide objective information to practitioners to arrive at an informed estimate. In this regard, we have developed a hybrid method, that utilizes change impact analysis information for improving effort estimation. We also developed an estimation model based on gradient boosted trees (GBT). In this study, we evaluate the performance and usefulness of our hybrid method with tool support and the GBT model in a live iteration at Insiders Technologies GmbH, a German software company. Additionally, the solution was also assessed for perceived usefulness and understandability in a study with graduate and post-graduate students. The results from the industrial evaluation show that the proposed method produces more accurate estimates than only expert-based or only model-based estimates. Furthermore, both students and practitioners perceived the usefulness and understandability of the method positively.

  • 42.
    Tran, Huynh Khanh Vi
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Unterkalmsteiner, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Test-Case Quality: Understanding Practitioners’ Perspectives2019In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) / [ed] Franch X.,Mannisto T.,Martinez-Fernandez S., Springer , 2019, Vol. 11915, p. 37-52Conference paper (Refereed)
    Abstract [en]

    Background: Test-case quality has always been one of the major concerns in software testing. To improve test-case quality, it is important to better understand how practitioners perceive the quality of test-cases. Objective: Motivated by that need, we investigated how practitioners define test-case quality and which aspects of test-cases are important for quality assessment. Method: We conducted semi-structured interviews with professional developers, testers and test architects from a multinational software company in Sweden. Before the interviews, we asked participants for actual test cases (written in natural language) that they perceive as good, normal, and bad respectively together with rationales for their assessment. We also compared their opinions on shared test cases and contrasted their views with the relevant literature. Results: We present a quality model which consists of 11 test-case quality attributes. We also identify a misalignment in defining test-case quality among practitioners and between academia and industry, along with suggestions for improving test-case quality in industry. Conclusion: The results show that practitioners’ background, including roles and working experience, are critical dimensions of how test-case quality is defined and assessed. © Springer Nature Switzerland AG 2019.

  • 43.
    Tran, Huynh Khanh Vi
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Unterkalmsteiner, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    How good are my search strings? Reflections on using an existing review as a quasi-gold standard2022In: e-Informatica Software Engineering Journal, ISSN 1897-7979, E-ISSN 2084-4840, Vol. 16, no 1, p. 69-89, article id 220103Article, review/survey (Refereed)
    Abstract [en]

    Background: Systematic literature studies (SLS) have become a core research methodology in Evidence-based Software Engineering (EBSE). Search completeness, i.e., finding all relevant papers on the topic of interest, has been recognized as one of the most commonly discussed validity issues of SLSs. Aim: This study aims at raising awareness on the issues related to search string construction and on search validation using a quasi-gold standard (QGS). Furthermore, we aim at providing guidelines for search string validation. Method: We use a recently completed tertiary study as a case and complement our findings with the observations from other researchers studying and advancing EBSE. Results: We found that the issue of assessing QGS quality has not seen much attention in the literature, and the validation of automated searches in SLSs could be improved. Hence, we propose to extend the current search validation approach by the additional analysis step of the automated search validation results and provide recommendations for the QGS construction. Conclusion: In this paper, we report on new issues which could affect search completeness in SLSs. Furthermore, the proposed guideline and recommendations could help researchers implement a more reliable search strategy in their SLSs.

  • 44.
    Tran, Huynh Khanh Vi
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Unterkalmsteiner, Michael
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Börstler, Jürgen
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Assessing test artifact quality-A tertiary study2021In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 139, article id 106620Article in journal (Refereed)
    Abstract [en]

    Context: Modern software development increasingly relies on software testing for an ever more frequent delivery of high quality software. This puts high demands on the quality of the central artifacts in software testing, test suites and test cases. Objective: We aim to develop a comprehensive model for capturing the dimensions of test case/suite quality, which are relevant for a variety of perspectives. Methods: We have carried out a systematic literature review to identify and analyze existing secondary studies on quality aspects of software testing artifacts. Results: We identified 49 relevant secondary studies. Of these 49 studies, less than half did some form of quality appraisal of the included primary studies and only 3 took into account the quality of the primary study when synthesizing the results. We present an aggregation of the context dimensions and factors that can be used to characterize the environment in which the test case/suite quality is investigated. We also provide a comprehensive model of test case/suite quality with definitions for the quality attributes and measurements based on findings in the literature and ISO/IEC 25010:2011. Conclusion: The test artifact quality model presented in the paper can be used to support test artifact quality assessment and improvement initiatives in practice. Furthermore, the model can also be used as a framework for documenting context characteristics to make research results more accessible for research and practice.

    Download full text (pdf)
    fulltext
  • 45.
    Usman, Muhammad
    et al.
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Ali, Nauman bin
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    Wohlin, Claes
    Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
    A Quality Assessment Instrument for Systematic Literature Reviews in Software Engineering2023In: e-Informatica Software Engineering Journal, ISSN 1897-7979, E-ISSN 2084-4840, Vol. 17, no 1, article id 230105Article in journal (Refereed)
    Abstract [en]

    Background: Systematic literature reviews (SLRs) have become a standard practice as part of software engineering (SE) research, although their quality varies. To build on the reviews, both for future research and industry practice, they need to be of high quality.Aim: To assess the quality of SLRs in SE, we put forward an appraisal instrument for SLRs.Method: A well-established appraisal instrument from research in healthcare was used as a starting point to develop the instrument. It is adapted to SE using guidelines, checklists, and experiences from SE. The first version was reviewed by four external experts on SLRs in SE and updated based on their feedback. To demonstrate its use, the updated version was also used by the authors to assess a sample of six selected systematic literature studies.Results: The outcome of the research is an appraisal instrument for quality assessment of SLRs in SE. The instrument includes 15 items with different options to capture the quality. The instrument also supports consolidating the items into groups, which are then used to assess the overall quality of an SLR.Conclusion: The presented instrument may be helpful support for an appraiser in assessing the quality of SLRs in SE.

    Download full text (pdf)
    fulltext
1 - 45 of 45
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf