Background: Developing and maintaining software is resource expensive. Therefore many organizations and practitioners use Open Source Software(OSS) in commercial development to minimize resource expense. But, adopting OSS mandates the organizations and practitioners to assess and monitor the OSS for updates from the community. Previous literature proposes many assessment frameworks. Many are non-automated and use complex attributes that require a steep learning curve for practitioners to understand. The OSS assessments and monitoring choke the agility of the team and delay their time to market. Practitioners need automated quick assessments with easy-to-understand attributes to assist them during OSS adoption. After adoption, monitoring and upgrading OSS can be challenging. Therefore, organizations need automatic OSS monitoring and upgrading solutions capable of checking community updates for the OSS and upgrade the internally hosted OSS if the update is compliant automatically.

Objectives: The objective of our thesis is to automate OSS assessments and monitoring using OSS assessment attributes that are easily understood by the practitioners.

Methods: We performed a design science research at City Network to understand OSS assessments and monitoring in organizations and identified the attributes used by the practitioners. Additionally, we identified the attributes from the previous literature that were relevant for practitioners. Following the identification, we constructed an automated solution for OSS assessments and monitoring that was accepted by City Network. To evaluate the generalizability of our automated solution, we conducted interviews with practitioners outside City Network.

Results: Our automated solution was praised for its ease of use and easy-to-understand attributes. Practitioners wanted their customizations on our automated solution with additional features and attributes to fully automate their OSS assessments. But our OSS monitoring and upgrading solution was criticized for lack of rigor in assessing an update. But, its program flow and usage at scale were appreciated by practitioners.

Conclusions: Our automated solution was effective in automated assessing OSS before adoption. But, it was not capable of automating OSS monitoring and upgrading. With that said, the problem of OSS assessments and monitoring is relevant for many organizations and practitioners. Therefore, such research is essential to improve and streamline OSS adoption for organizations and practitioners. Additionally, it is worthwhile to research more OSS attributes that are relevant and easy to understand for the practitioners.