Files
Olewicki_39641400_2020.pdf
UCLouvain restricted access - Adobe PDF
- 1.24 MB
Details
- Supervisors
- Faculty
- Degree label
- Abstract
- Continuous Integration (CI) has become the core infrastructure of modern software organizations, enabling them to update projects more regularly (i.e., daily) while keeping the quality of the code valid and relevant. However, with more than 100,000 automated build jobs triggered each day by, for example, the Mozilla CI, build automation comes with a financial and time cost that should not be neglected. Through this thesis, we aim at validating the hypothesis that the cost of build automation is not negligible and that the build activity could be reduced. This thesis presents the results of a quantitative study of the CI over the course of 20 months, analyzing over 54 million Mozilla build jobs to identify the cost of build automation triggered for each code change as well as a build reduction proposition, focusing on the case of brown builds at Ubisoft. We find that the median build cost of a commit on the Mozilla Firefox project increases from 19-45 USD for release 36 to 79-125 USD for release 50. Less than 10% of that cost is incurred for builds that identified failures or validated a build fix, showing that the cost of automation could be significantly reduced by applying build selection or prioritization techniques on the other 90% of the build cost. Furthermore, some builds have inconsistent results, i.e., not every failure is a real failure. Those unreliable builds are identified as brown builds. As a result, developers lose trust in the CI system and manually rerun failing builds without correcting potential errors in the code changes they pushed into their project. Rerunning automatically brown builds or correcting the brownness would improve the CI system and reduce build activity by improving the reliability of the system. This thesis introduces trained and validated models that provided early detection of brown build results with a mean accuracy of 84%, a mean precision of 46% and a mean recall of 64%.