Dr. Dengxin Dai

Senior Researcher
E-Mail Please contact TODO
Phone +41 xx xxx xx xx
Address Stampfenbachstrasse 48, 8092 Zürich, Switzerland
Room ETH Zurich, Computer Vision Lab

Biography

Dengxin Dai is a Senior Researcher working with the Computer Vision Lab at ETH Zurich. In 2016, he obtained his PhD in Computer Vision at ETH Zurich. Since then he is the Team Leader of TRACE-Zurich, working on Autonomous Driving within the R&D project “TRACE: Toyota Research on Automated Cars in Europe”. His research interests lie in autonomous driving, robust perception in adverse weather and illumination conditions, automotive sensors and computer vision under limited supervision.

Research Interests & Workshops

He has organized a CVPR’19 Workshop on Vision for All Seasons: Bad Weather and Nighttime , and is organizing an ICCV’19 workshop on Autonomous Driving. He has been a program committee member of several major computer vision conferences and received multiple outstanding reviewer awards. He is a guest editor for the IJCV special issue Vision for All Seasons and is an area chair for WACV 2020.

Publications & Projects

  • Test-time Domain Adaptation for Monocular Depth Estimation

    Authors: Zhi Li, Shaoshuai Shi, Bernt Schiele, Dengxin Dai

    ICRA, 2023

    a novel test-time domain adaptation method for depth estimation

    Test-time domain adaptation, i.e. adapting source pretrained models to the test data on-the-fly in a source-free, unsupervised manner, is a highly practical yet very challenging task. Due to the domain gap between source and target data, inference quality on the target domain can drop drastically especially in terms of absolute scale of depth. In addition, unsupervised adaptation can degrade the model performance due to inaccurate pseudo labels. Furthermore, the model can suffer from catastrophic forgetting when errors are accumulated over time. We propose a test-time domain adaptation framework for monocular depth estimation which achieves both stability and adaptation performance by benefiting from both self-training of the supervised branch and pseudo labels from self-supervised branch, and is able to tackle the above problems: our scale alignment scheme aligns the input features between source and target data, correcting the absolute scale inference on the target domain; with pseudo label consistency check, we select confident pixels thus improve pseudo label quality; regularisation and self-training schemes are applied to help avoid catastrophic forgetting. Without requirement of further supervisions on the target domain, our method adapts the source-trained models to the test data with significant improvements over the direct inference results, providing scalea-ware depth map outputs that outperform the state-of-the-arts.
    Read More

  • Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts

    Authors: Qi Fan, Mattia Segu, Yu-Wing Tai, Fisher Yu, Chi-Keung Tang, Bernt Schiele, Dengxin Dai

    ICLR, 2023

    a very simple yet effective method for object detection across domains

    Improving model's generalizability against domain shifts is crucial, especially for safety-critical applications such as autonomous driving. Real-world domain styles can vary substantially due to environment changes and sensor noises, but deep models only know the training domain style. Such domain style gap impedes model generalization on diverse real-world domains. Our proposed Normalization Perturbation (NP) can effectively overcome this domain style overfitting problem. We observe that this problem is mainly caused by the biased distribution of low-level features learned in shallow CNN layers. Thus, we propose to perturb the channel statistics of source domain features to synthesize various latent styles, so that the trained deep model can perceive diverse potential domains and generalizes well even without observations of target domain data in training. We further explore the style-sensitive channels for effective style synthesis. Normalization Perturbation only relies on a single source domain and is surprisingly effective and extremely easy to implement. Extensive experiments verify the effectiveness of our method for generalizing models under real-world domain shifts.
    Read More

  • Benchmarking the Robustness of LiDAR Semantic Segmentation Models

    Authors: Xu Yan, Chaoda Zheng, Zhen Li, Shuguang Cui, and Dengxin Dai

    IJCV (under review), 2022

    a comprehensive study on the robustness of LiDAR semantic segmentation methods

    When using LiDAR semantic segmentation models for safety-critical applications such as autonomous driving, it is essential to understand and improve their robustness with respect to a large range of LiDAR corruptions. In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. To rigorously evaluate the robustness and generalizability of current approaches, we propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. Then, we systematically investigate 11 LiDAR semantic segmentation models, especially spanning different input representations (e.g., point clouds, voxels, projected images, and etc.), network architectures and training schemes. Through this study, we obtain two insights: 1) We find out that the input representation plays a crucial role in robustness. Specifically, under specific corruptions, different representations perform variously. 2) Although state-of-the-art methods on LiDAR semantic segmentation achieve promising results on clean data, they are less robust when dealing with noisy data. Finally, based on the above observations, we design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications. It is promising that our benchmark, comprehensive analysis, and observations can boost future research in robust LiDAR semantic segmentation for safety-critical applications.
    Read More

  • Learnable Online Graph Representations for 3D Multi-Object Tracking

    Authors: Jan-Nico Zaech, Dengxin Dai, Alexander Liniger, Martin Danelljan, Luc Van Gool

    ICRA, 2022

    a unified and learning based approach to the 3D MOT problem

    Tracking of objects in 3D is a fundamental task in computer vision that finds use in a wide range of applications such as autonomous driving, robotics or augmented reality. Most recent approaches for 3D multi object tracking (MOT) from LIDAR use object dynamics together with a set of handcrafted features to match detections of objects. However, manually designing such features and heuristics is cumbersome and often leads to suboptimal performance. In this work, we instead strive towards a unified and learning based approach to the 3D MOT problem. We design a graph structure to jointly process detection and track states in an online manner. To this end, we employ a Neural Message Passing network for data association that is fully trainable. Our approach provides a natural way for track initialization and handling of false positive detections, while significantly improving track stability. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
    Read More

  • Pix2NeRF: Unsupervised Conditional p-GAN for Single Image to Neural Radiance Fields Translation

    Authors: Shengqu Cai, Anton Obukhov, Dengxin Dai, and Luc Van Gool

    CVPR, 2022

    a novel pipeline to generate Neural Radiance Fields (NeRF) of an object or a scene of a specific class, conditioned on a single input image

    We propose a pipeline to generate Neural Radiance Fields (NeRF) of an object or a scene of a specific class, conditioned on a single input image. This is a challenging task, as training NeRF requires multiple views of the same scene, coupled with corresponding poses, which are hard to obtain. Our method is based on pi-GAN, a generative model for unconditional 3D-aware image synthesis, which maps random latent codes to radiance fields of a class of objects. We jointly optimize (1) the pi-GAN objective to utilize its high-fidelity 3D-aware generation and (2) a carefully designed reconstruction objective. The latter includes an encoder coupled with pi-GAN generator to form an auto-encoder. Unlike previous few-shot NeRF approaches, our pipeline is unsupervised, capable of being trained with independent images without 3D, multi-view, or pose supervision. Applications of our pipeline include 3d avatar generation, object-centric novel view synthesis with a single input image, and 3d-aware super-resolution, to name a few.
    Read More

  • CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning

    Authors: Yue Fan, Dengxin Dai, Anna Kukleva and Bernt Schiele

    CVPR, 2022

    a novel co-learning framework (CoSSL) with decoupled representation learning and classifier learning for imbalanced SSL

    Standard semi-supervised learning (SSL) using class-balanced datasets has shown great progress to leverage unlabeled data effectively. However, the more realistic setting of class-imbalanced data - called imbalanced SSL - is largely underexplored and standard SSL tends to underperform. In this paper, we propose a novel co-learning framework (CoSSL), which decouples representation and classifier learning while coupling them closely. To handle the data imbalance, we devise Tail-class Feature Enhancement (TFE) for classifier learning. Furthermore, the current evaluation protocol for imbalanced SSL focuses only on balanced test sets, which has limited practicality in real-world scenarios. Therefore, we further conduct a comprehensive evaluation under various shifted test distributions. In experiments, we show that our approach outperforms other methods over a large range of shifted distributions, achieving state-of-the-art performance on benchmark datasets ranging from CIFAR-10, CIFAR-100, ImageNet, to Food-101. Our code will be made publicly available.
    Read More

  • End-to-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization

    Authors: Niclas Vödisch, Ozan Unal, Ke Li, Luc Van Gool, and Dengxin Dai

    ICRA, 2022

    The first end-to-end approach to learn to optimize the LiDAR beam configuration for given applications

    Existing learning methods for LiDAR-based applications use 3D points scanned under a pre-determined beam configuration, e.g., the elevation angles of beams are often evenly distributed. Those fixed configurations are task-agnostic, so simply using them can lead to sub-optimal performance. In this work, we take a new route to learn to optimize the LiDAR beam configuration for a given application. Specifically, we propose a reinforcement learning-based learning-to-optimize (RL-L2O) framework to automatically optimize the beam configuration in an end-to-end manner for different LiDAR-based applications. The optimization is guided by the final performance of the target task and thus our method can be integrated easily with any LiDAR-based application as a simple drop-in module. The method is especially useful when a low-resolution (low-cost) LiDAR is needed, for instance, for system deployment at a massive scale. We use our method to search for the beam configuration of a low-resolution LiDAR for two important tasks: 3D object detection and localization. Experiments show that the proposed RL-L2O method improves the performance in both tasks significantly compared to the baseline methods. We believe that a combination of our method with the recent advances of programmable LiDARs can start a new research direction for LiDAR-based active perception. The code is publicly available at github.com/vniclas/lidar_beam_selection.
    Read More

  • Adiabatic Quantum Computing for Multi Object Tracking

    Authors: Jan-Nico Zaech, Alexander Liniger, Martin Danelljan, Dengxin Dai and Luc Van Gool

    CVPR, 2022

    The first MOT formulation designed to be solved with Adiabatic Quantum Computing.

    Multi-Object Tracking (MOT) is most often approached in the tracking-by-detection paradigm, where object detections are associated through time. The association step naturally leads to discrete optimization problems. As these optimization problems are often NP-hard, they can only be solved exactly for small instances on current hardware. Adiabatic quantum computing (AQC) offers a solution for this, as it has the potential to provide a considerable speedup on a range of NP-hard optimization problems in the near future. However, current MOT formulations are unsuitable for quantum computing due to their scaling properties. In this work, we therefore propose the first MOT formulation designed to be solved with AQC. We employ an Ising model that represents the quantum mechanical system implemented on the AQC. We show that our approach is competitive compared with state-of-the-art optimization-based approaches, even when using of-the-shelf integer programming solvers. Finally, we demonstrate that our MOT problem is already solvable on the current generation of real quantum computers for small examples, and analyze the properties of the measured solutions.
    Read More

  • LiDAR Snowfall Simulation for Robust 3D Object Detection

    Authors: Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu Dengxin Dai and Luc Van Gool

    CVPR (Oral), 2022

    A novel simulation approach to simulate snowfall effects into existing LiDAR dataset to train robust LiDAR-based perception methods for adverse weather

    3D object detection is a central task for applications such as autonomous driving, in which the system needs to localize and classify surrounding traffic agents, even in the presence of adverse weather. In this paper, we address the problem of LiDAR-based 3D object detection under snowfall. Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds. Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the measurement for each LiDAR beam accordingly. Moreover, as snowfall often causes wetness on the ground, we also simulate ground wetness on LiDAR point clouds. We use our simulation to generate partially synthetic snowy LiDAR data and leverage these data for training 3D object detection models that are robust to snowfall. We conduct an extensive evaluation using several state-of-the-art 3D object detection methods and show that our simulation consistently yields significant performance gains on the real snowy STF dataset compared to clear-weather baselines and competing simulation approaches, while not sacrificing performance in clear weather. Our code is available at github.com/SysCV/LiDAR_snow_sim.
    Read More

  • Continual Test-Time Domain Adaptation

    Authors: Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai

    CVPR, 2022

    A novel method for long-term test-time adaptation under continually changing environments.

    Test-time domain adaptation aims to adapt a source pre-trained model to a target domain without using any source data. Existing works mainly consider the case where the target domain is static. However, real-world machine perception systems are running in non-stationary and continually changing environments where the target domain distribution can change over time. Existing methods, which are mostly based on self-training and entropy regularization, can suffer from these non-stationary environments. Due to the distribution shift over time in the target domain, pseudo-labels become unreliable. The noisy pseudo-labels can further lead to error accumulation and catastrophic forgetting. To tackle these issues, we propose a continual test-time adaptation approach (CoTTA) which comprises two parts. Firstly, we propose to reduce the error accumulation by using weight-averaged and augmentation-averaged predictions which are often more accurate. On the other hand, to avoid catastrophic forgetting, we propose to stochastically restore a small part of the neurons to the source pre-trained weights during each iteration to help preserve source knowledge in the long-term. The proposed method enables the long-term adaptation for all parameters in the network. CoTTA is easy to implement and can be readily incorporated in off-the-shelf pre-trained models. We demonstrate the effectiveness of our approach on four classification tasks and a segmentation task for continual test-time adaptation, on which we outperform existing methods. Our code is available at https://qin.ee/cotta.
    Read More

  • Test-time Domain Adaptation for Monocular Depth Estimation

    task. Due to the domain gap between source and target data, inference quality on the target domain can drop [...]

    Read More

  • Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts

    Improving model's generalizability against domain shifts is crucial, especially for safety-critical applications such as autonomous driving. Real-world domain styles [...]

    Read More

  • Benchmarking the Robustness of LiDAR Semantic Segmentation Models

    Benchmarking the Robustness of LiDAR Semantic Segmentation Models Xu Yan, Chaoda Zheng, Zhen Li, Shuguang Cui, Dengxin Dai When using LiDAR semantic [...]

    Read More

  • Learnable Online Graph Representations for 3D Multi-Object Tracking

    Learnable Online Graph Representations for 3D Multi-Object Tracking JN Zaech, D Dai, A Liniger, M Danelljan, L Van Gool [...]

    Read More

  • Pix2NeRF: Unsupervised Conditional p-GAN for Single Image to Neural Radiance Fields Translation

    Pix2NeRF: Unsupervised Conditional p-GAN for Single Image to Neural Radiance Fields Translation Shengqu Cai, Anton Obukhov, Dengxin Dai, Luc [...]

    Read More

  • CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning

    Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning Yue Fan, Dengxin Dai, Anna Kukleva, Bernt Schiele Standard [...]

    Read More

  • End-to-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization

    End-to-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization Niclas Vödisch, Ozan Unal, Ke Li, Luc [...]

    Read More

  • Adiabatic Quantum Computing for Multi Object Tracking

    Adiabatic Quantum Computing for Multi Object Tracking Jan-Nico Zaech, Alexander Liniger, Martin Danelljan, Dengxin Dai, Luc Van Gool [...]

    Read More

  • LiDAR Snowfall Simulation for Robust 3D Object Detection

    LiDAR Snowfall Simulation for Robust 3D Object Detection Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin [...]

    Read More

  • Continual Test-Time Domain Adaptation

    Continual Test-Time Domain Adaptation Qin Wang, Olga Fink, Luc Van Gool, Dengxin Dai Test-time domain adaptation aims to [...]

    Read More