Designing an extensive quality assessment framework for open source software projects

Abstract

In the present work, Open Source Software - Quality Assessment and Ranking Engine (OSS-QARE) framework has been proposed to assess the quality of OSS projects at extensive level. The framework operates through a structured process, divided into four distinct phases - Identification of OSS projects for analysis, Definition of assessment criteria, Data extraction, storage and metrics computation; and assessment, ranking and OSS recommendation. The OSS-QARE framework incorporates the strengths of existing models and introduces innovations where existing models fall short. The assessment criterion quantifies the OSS quality by dividing it into a set of characteristics, sub-characteristics and metrics in a hierarchical manner. Metrics used in the OSS-QARE framework are computed on the data stored in the local database. The comparative quality scores are computed using multi criteria decision making. The OSS-QARE framework has been implemented in the form of a software application referred to as OSS-QARE tool. The tool automates the process of data fetching, data pre-processing, data storage, metrics computation and final quality score assessment. The tool utilizes the GitHub API to fetch real-time data for quality assessment and establishes the database schema and related tables to store this information. It provides an overall comparative ranking along with micro level quality parameter scores for the OSS projects being assessed. The OSS-QARE framework has been validated by applying it on seven OSS projects and comparing the results with the outcomes produced by an existing model. In addition to producing quality scores in conformance with the existing quality evaluation model, the OSS-QARE framework provides an opportunity for a detailed investigation of the quality scores obtained by the OSS projects. The functionality and application of OSS-QARE framework over diverse domains has been illustrated by taking GitHub as a case study hosting platform.