Autonomous Robots Lab
  • Home
  • News
  • Research
    • Autonomous Navigation and Exploration
    • Fixed-Wing UAVs
    • Agile and Physical Interaction Control
    • Localization and 3D Reconstruction
    • Subterranean Robotics
    • Collision-tolerant Aerial Robots
    • Marine Robotics
    • Intelligent Mobility >
      • Student Projects
      • Electric Bus Datasets
    • Robotics for Nuclear Sites
    • Degerminator
    • Autonomous Robots Arena
    • Code
    • Media
    • Research Presentations
    • Projects
  • Publications
  • Group
    • People
    • Research Collaborators
    • Positions
  • Education
    • Introduction to Aerial Robotics >
      • Online Textbook >
        • Modeling >
          • Frame Rotations and Representations
          • Multirotor Dynamics
        • State Estimation >
          • Inertial Sensors
          • Batch Discrete-Time Estimation
          • The Kalman Filter
        • Flight Control >
          • PID Control
          • LQR Control
          • Linear Model Predictive Control
        • Motion Planning >
          • Holonomic Vehicle BVS
          • Dubins Airplane
          • Collision-free Navigation
          • Structural Inspection Path Planning
        • Simulation Tools >
          • Simulations with SimPy
          • MATLAB & Simulink
          • RotorS Simulator >
            • RotorS Simulator Video Examples
      • Lecture Slides
      • Literature and Links
      • RotorS Simulator
      • Student Projects
      • Homework Assignments
      • Independent Study
      • Video Explanations
      • Syllabus
      • Grade Statistics
    • Autonomous Mobile Robot Design >
      • Lecture Slides
      • Semester Projects
      • Code Repository
      • Literature and Links
      • RotorS Simulator
      • Video Explanations
      • Resources for Semester Projects
      • Syllabus
    • Robotics for DDD Applications
    • CS302 - Data Structures
    • Student Projects >
      • Robot Competitions
      • Undergraduate Researchers Needed
      • ConstructionBots - Student Projects
    • EiT TTK4854 - Robotic Ocean Waste Removal
    • Aerial Robotic Autonomy >
      • Breadth Topics
      • Deep-dive Topics
      • Literature
    • Robotics Seminars
    • Robotics Days
    • Outreach >
      • Drones Demystified! >
        • Lecture Slides
        • Code Repository
        • Video Explanations
        • RotorS Simulator
        • Online Textbook
      • Autonomous Robots Camp >
        • RotorS Simulator
      • Outreach Student Projects
    • BadgerWorks >
      • General Study Links
      • Learn ROS
      • SubT-Edu
  • Resources
    • Autonomous Robots Arena
    • Robot Development Space
  • Contact
Keyframe-based Thermal-Inertial Odometry 
Shehryar Khattak            Christos Papachristos                  Kostas Alexis
Autonomous Robots Lab, University of Nevada, Reno
Picture
Abstract: ​Autonomous navigation of Micro Aerial Vehicles in environments that are simultaneously GPS-denied and visually-degraded, and especially in dark, texture-less and dust- or smoke-filled settings is rendered particularly hard. However, a potential solution arises if such aerial robots are equipped with Long Wave Infrared thermal vision systems which are unaffected by darkness and can penetrate most types of obscurants. In response to this fact, this work proposes a keyframe-based thermal--inertial odometry estimation framework tailored to the exact data and concepts of operation of thermal cameras. The front-end component of the proposed solution utilizes full radiometric data to establish reliable correspondences between thermal images, as opposed to operating on rescaled images as previous efforts have presented. In parallel, taking advantage of a keyframe-based optimization back-end the proposed method is suitable for handling periods of data interruption which are commonly present in thermal cameras, while it also ensures the joint optimization of re-projection errors of 3D landmarks and inertial measurement errors. The developed framework was verified with respect to its resilience, performance and ability to enable autonomous navigation in an extensive set of experimental studies including multiple field deployments in severely degraded, dark and obscurants-filled underground mines.

Publication

Under Review

Video Results

Inactive Underground Mine & Vicon Room
Active Underground Mine & Urban Parking Lot

Code

Coming soon! 

Highlights from the Underground Thermal Data

To emphasize the importance of operating on full radiometric (14-bit or 16-bit) thermal data, below we present two versions of the rescaled thermal imagey from an underground field deployment in the Comstock Exploratory Mine in Virginia City, Nevada, US. The first version (left) is based on histogram equalization per frame (analogous to AGC typically introduced by thermal camera manufacturers). The second version is based on fixed-range normalization.​
Indicative time instance: 
  • 1:55 - 2:15 An example of particularly low contrast thermal imagery as robot turns a corner in the mine while facing towards the wall, presenting a uniform temperature surface.
Indicative time instance: 
  • ​0:40 - 0:55 An example of rapid and significant appearance change between successive images.

Dataset

This dataset contains thermal images either a FLIR Tau2 or a FLIR Boson (16-bit format), VN100 IMU measurements and in some cases pointclouds from a Velodyne PUCK Lite LiDAR or an Ouster OS1-64. The provided datasets were collected during the autonomous flight experiments of our robot using an on-board computer running Ubuntu 16.04 and ROS Kinetic at a) the Comstock underground mine in Northern Nevada, b) an active gold mine in Northern Nevada, c) an Urban Parking Lot, as well as d) our Autonomous Robots Arena. All datasets contain both sensor data, as well as intrinsic and extrinsic sensor calibrations (YAML files). 
​
 
Relevant Downloads
  • Comstock Dataset (folder includes calibration files) 
  • Active Gold Mine Dataset (folder includes calibration files)
  • Urban Parking Lot Dataset (folder includes calibration files)
  • Arena (VICON) Dataset (folder includes calibration files) 
Proudly powered by Weebly