Deep learning based scene flow estimation with point clouds

Details

Supervisors

Schaus, Pierre

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] en sciences informatiques, à finalité spécialisée

Abstract

The Scene flow, defined as ‘the three-dimensional motion field of points in the world’, is an essential component for mobile robotics and autonomous vehicles, capturing the motion of dynamic objects in our surroundings and serving as a basis for higher-level scene understanding. With 3D sensors becoming more affordable, such as LiDARs and RGB-D cameras, and with the recent progress in deep learning, deep neural networks directly processing raw point clouds have been the subject of growing interest for several years now for scene understanding tasks. However, such methods usually do not take into account the real-time nature of LiDAR point clouds, scanned at a frequency of 10Hz. They remove the ground points naïvely using a height threshold, potentially eliminating moving points, and generally do not make any distinction between the foreground and background, which are moving and static, respectively. The ego-motion of the car, i.e. the rotation and translation of the car between two time frames, is taken into account, hence not reflecting the true motion of points in the environment. This thesis proposes a simple ground segmentation algorithm, relying on standard operations on point clouds, followed by the cancellation of the ego-motion using a registration method. A clustering-based classification of objects in static and moving classes is then presented, further eliminating static points. Finally, a network hierarchically predicting the flows of moving points is proposed, exploring a new feature learning and downsampling method.