The inevitable thermodynamic costs of neural networks

Details

Supervisors

Delvenne, Jean-Charles

Degree label

Master [120] : ingénieur civil en science des données, à finalité spécialisée

Abstract

Whatever the quality of the hardware, the laws of thermodynamics will limit the energy efficiency we can achieve when making computation. Today, we are still unaware of the extent of these limits in the field of machine learning. Can we compute smarter to reduce these limits? In this work, we propose a Python library that quantifies these limitations for Artificial Neural Networks(ANN), a specific branch of machine learning. Based on numerical samples, we propose an estimate of a lower bound on the minimum energy cost, namely the Landauer and Mismatch cost, required to run an already trained neural network. Using these results, we suggest a set of neural structure recommendations that will reduce the lower bound for such networks.