Keyframe-based Thermal-Inertial Odometry
Shehryar Khattak Christos Papachristos Kostas Alexis
Autonomous Robots Lab, University of Nevada, Reno
Abstract: Autonomous navigation of Micro Aerial Vehicles in environments that are simultaneously GPS-denied and visually-degraded, and especially in dark, texture-less and dust- or smoke-filled settings is rendered particularly hard. However, a potential solution arises if such aerial robots are equipped with Long Wave Infrared thermal vision systems which are unaffected by darkness and can penetrate most types of obscurants. In response to this fact, this work proposes a keyframe-based thermal--inertial odometry estimation framework tailored to the exact data and concepts of operation of thermal cameras. The front-end component of the proposed solution utilizes full radiometric data to establish reliable correspondences between thermal images, as opposed to operating on rescaled images as previous efforts have presented. In parallel, taking advantage of a keyframe-based optimization back-end the proposed method is suitable for handling periods of data interruption which are commonly present in thermal cameras, while it also ensures the joint optimization of re-projection errors of $3\textrm{D}$ landmarks and inertial measurement errors. The developed framework was verified with respect to its resilience, performance and ability to enable autonomous navigation in an extensive set of experimental studies including multiple field deployments in severely degraded, dark and obscurants-filled underground mines.
Publication
Under Review
Video
Code
Coming soon!
Highlights from the Underground Thermal Data
To emphasize the importance of operating on full radiometric (14-bit or 16-bit) thermal data, below we present two versions of the rescaled thermal imagey from an underground field deployment in the Comstock Exploratory Mine in Virginia City, Nevada, US. The first version (left) is based on histogram equalization per frame (analogous to AGC typically introduced by thermal camera manufacturers). The second version is based on fixed-range normalization.
|
Indicative time instance:
|
Indicative time instance:
|
Dataset
This dataset contains thermal images from FLIR Boson (16-bit format), VN100 IMU measurements and pointclouds from Velodyne PUCK Lite LiDAR. The provided dataset was collected during the autonomous flight experiments of our robot using an on-board computer running Ubuntu 16.04 and ROS Kinetic at the Comstock underground mine in Northern Nevada. A list of rosbag published topics and their brief description is provided below: