Keyframe-based Thermal-Inertial Odometry
Shehryar Khattak Christos Papachristos Kostas Alexis
Autonomous Robots Lab, University of Nevada, Reno
Abstract: Autonomous navigation of Micro Aerial Vehicles in environments that are simultaneously GPS-denied and visually-degraded, and especially in dark, texture-less and dust- or smoke-filled settings is rendered particularly hard. However, a potential solution arises if such aerial robots are equipped with Long Wave Infrared thermal vision systems which are unaffected by darkness and can penetrate most types of obscurants. In response to this fact, this work proposes a keyframe-based thermal--inertial odometry estimation framework tailored to the exact data and concepts of operation of thermal cameras. The front-end component of the proposed solution utilizes full radiometric data to establish reliable correspondences between thermal images, as opposed to operating on rescaled images as previous efforts have presented. In parallel, taking advantage of a keyframe-based optimization back-end the proposed method is suitable for handling periods of data interruption which are commonly present in thermal cameras, while it also ensures the joint optimization of re-projection errors of 3D landmarks and inertial measurement errors. The developed framework was verified with respect to its resilience, performance and ability to enable autonomous navigation in an extensive set of experimental studies including multiple field deployments in severely degraded, dark and obscurants-filled underground mines.
Publication
Under Review
Video Results
Inactive Underground Mine & Vicon Room
|
Active Underground Mine & Urban Parking Lot
|
Code
Coming soon!
Highlights from the Underground Thermal Data
To emphasize the importance of operating on full radiometric (14-bit or 16-bit) thermal data, below we present two versions of the rescaled thermal imagey from an underground field deployment in the Comstock Exploratory Mine in Virginia City, Nevada, US. The first version (left) is based on histogram equalization per frame (analogous to AGC typically introduced by thermal camera manufacturers). The second version is based on fixed-range normalization.
Indicative time instance:
|
Indicative time instance:
|
Dataset
This dataset contains thermal images either a FLIR Tau2 or a FLIR Boson (16-bit format), VN100 IMU measurements and in some cases pointclouds from a Velodyne PUCK Lite LiDAR or an Ouster OS1-64. The provided datasets were collected during the autonomous flight experiments of our robot using an on-board computer running Ubuntu 16.04 and ROS Kinetic at a) the Comstock underground mine in Northern Nevada, b) an active gold mine in Northern Nevada, c) an Urban Parking Lot, as well as d) our Autonomous Robots Arena. All datasets contain both sensor data, as well as intrinsic and extrinsic sensor calibrations (YAML files).
Relevant Downloads
- Comstock Dataset (folder includes calibration files)
- Active Gold Mine Dataset (folder includes calibration files)
- Urban Parking Lot Dataset (folder includes calibration files)
- Arena (VICON) Dataset (folder includes calibration files)