Transformers for EEG classification : architectures, pre-training, and applications to epileptic seizure forecasting

Details

Supervisors

Macq, Benoît

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil biomédical, à finalité spécialisée

Abstract

Used at first in the fields of Natural Language Processing (NLP) and Computer Vision (CV), the transformer neural network recently became popular for electroencephalogram (EEG) signals analysis. However, despite being appreciated for their ability to understand long-term dependencies and their great training speed derived from their parallelizability, transformers require a large amount of labelled data to train effectively. Such data is often scarce in the medical field, as annotations made by experts are costly. This is why self-supervised training, using unlabelled data, has to be performed beforehand. On top of providing a non-exhaustive list of transformer architectures used for EEG classification, we present a way to design several datasets from unlabeled EEG data, which can then be used to pre-train transformers to learn representations of EEG signals. We tested this method on an epileptic seizure forecasting task on the TUSZ dataset (https://doi.org/10.3389/fninf.2018.00083) using a Multi-channel Vision Transformer (MViT, https://doi.org/10.3390/biomedicines10071551). Our results suggest that models pre-trained with this approach not only train significantly faster, but also yield better performances than models that are not pre-trained. The code produced during this thesis is available on this repository: https://github.com/tbary/MasterThesis.git.

Citations

• APA

Bary, T. (2023). Transformers for EEG classification : architectures, pre-training, and applications to epileptic seizure forecasting. https://dial-mem.test.bib.ucl.ac.be/handle/2078.2/32884