Binarized neural networks : bat call classification

Details

Supervisors

Nijssen, Siegfried

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] en science des données, orientation technologies de l'information, à finalité spécialisée

Abstract

Neural networks are known to solve complex and varied tasks but unfortunately, they are very energy intensive. Indeed, the operations performed by these models are particularly hardware intensive, which make them difficult to use on less powerful computers. However, nowadays we are looking for new ways to reduce our energy consumption. This is the reason that led to the development of binary neural networks. These have the particularity of using only binary weights and outputs, which greatly simplifies the calculations. In fact, the use of binary values saves memory space compared to the initial use of 32 bits with a double precision value. This space saving also greatly speeds up the calculations as 32 weights can be used directly instead of one. This advantage of binary neural networks means that they do not place excessive demands on the hardware of the computer on which they are run. In this thesis, we will see a case of application of binary neural networks to classify bat calls, the aim being to verify if the use of a binary neural network is possible for this task. Indeed, by their definition, they tend to perform less well than their counterparts that use double precision weights and outputs. We will therefore look at different architectures based on a Convolution Network (CNN) and compare their results. We will also look at different techniques to reduce the memory space of a machine learning model other than a neural network such as XGBoost. Finally, a binary neural network model will be run on different Raspberry PI to observe its execution time. This thesis shows that it is possible to achieve an execution time of 3.83 seconds for a binary network using 4 threads with a 3 second audio file on a Raspberry Pi 3B, in comparison, the double precision network reaches 8.02 seconds. The memory size of the binary network is 14 KB, which is much smaller than the 225 KB for the double precision equivalent. However, it should be noted that there is an overall reduction of 2-4% compared to the double precision network. Now, it should be possible to reduce the gap but it would need to optimize more the architecture or to improve the data processing. So, we can imagine using binary neural networks to classify bat calls in real time directly in the field. In addition, since they use less energy, the batteries used to power the devices will run longer than with a double precision network.