Combining deep learning and logical constraints on ROAD-R

Details

Supervisors

Nijssen, Siegfried

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en informatique, à finalité spécialisée

Abstract

Deep neural networks (DNNs) have recently become increasingly popular due to their impressive performance in various tasks. However, their “black box" nature can lead to unexpected behaviors, contradicting task-specific background knowledge. To address this, researchers introduced ROAD-R, a dataset containing videos for road event detection in the context of autonomous driving, coupled with a set of requirements expressed as logical constraints. It was shown that exploiting these constraints enables the improvement of deep learning models' performance while ensuring compliance with the requirements. This master's thesis then has two objectives. Firstly, we investigate the feasibility of improving a pre-trained model's performance by fine-tuning it using the set of requirements, without the need for a complete re-training in order to save time in the process. Second, we want to improve post-processing methods that enforce the constraints to the predictions. Subsequent experiments involved training baseline models, fine-tuning them, and applying our post-processings to them using constrained loss and output techniques from the ROAD-R paper. Our results showed that while fine-tuning pre-trained models using the set of constraints did not provide significant improvements, our proposed post-processing methods, based on the model's confidence in its predictions, consistently outperformed the ones presented in the ROAD-R paper. Nonetheless, a more in-depth study of fine-tuning will have to be carried out in future work, as well as other uncovered aspects.