Learnable Online Graph Representations for 3D Multi-Object Tracking

JN Zaech, D Dai, A Liniger, M Danelljan, L Van Gool

Autonomous systems that operate in dynamic environments require robust object tracking in 3D as one of their key components. Most recent approaches for 3D multi-object tracking (MOT) from LIDAR use object dynamics together with a set of handcrafted features to match detections of objects across multiple frames. However, manually designing such features and heuristics is cumbersome and often leads to suboptimal performance. In this work, we instead strive towards a unified and learning based approach to the 3D MOT problem. We design a graph structure to jointly process detection and track states in an online manner. To this end, we employ a Neural Message Passing network for data association that is fully trainable. Our approach provides a natural way for track initialization and handling of false positive detections, while significantly improving track stability. We demonstrate the merit of the proposed approach in the nuScenes tracking challenge 2021 with a state-of-the-art performance of 65.6% AMOTA with 58% fewer ID-switches, resulting in the best LIDAR only submission and an overall second place.

Figure 1. The proposed method uses a graph representation for detections and tracks. A neural message passing based architecture performs matching of detections and tracks and provides a learning based framework for track initialization, effectively replacing heuristics that are required in current approaches.

Fig.2: The proposed tracking graph combines tracks, represented by a sequence of track nodes and detections in a single representation. During the NMP iterations, information is exchanged between nodes and edges, and thus, distributed globally throughout the graph.

Fig.3: Visualization of different update scenarios, with only active edges in the graph. The graph represents a single track and two detections at each time step. a) Shows the ideal case where a track is matched to one node at every timestep and each detection node is connected with each other. b) Represents the case where a match at one timestep is dropped and the track is only matched to two detection nodes. c) Shows a situation, where the proposed approach is able to decide for the globally best solution, even though two detection nodes have been matched to the track in the last frame.

Table 1: Results on the nuScenes test set. Methods marked with asterisk use private detections and thus, no direct comparison is possible.


Learnable Online Graph Representations for 3D Multi-Object Tracking

Jan-Nico Zaech, Dengxin Dai, Alexander Liniger, Martin Danelljan, Luc Van Gool

CVPR 2022