Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry

Masahiro Yamaguchi, Trong Phuc Truong, Shohei Mori, Vincent Nozick, Hideo Saito, Shoji Yachida, Hideaki Sato

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

In this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

Original languageEnglish
Pages (from-to)1296-1307
Number of pages12
JournalIEICE Transactions on Information and Systems
VolumeE101D
Issue number5
DOIs
Publication statusPublished - 2018 May 1

Fingerprint

Infrared radiation
Cameras
Hot Temperature
Triangulation

Keywords

  • 3D thermal map
  • Calibration
  • Delaunay division
  • Thermal-infrared camera
  • Visual odometry

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this

Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry. / Yamaguchi, Masahiro; Truong, Trong Phuc; Mori, Shohei; Nozick, Vincent; Saito, Hideo; Yachida, Shoji; Sato, Hideaki.

In: IEICE Transactions on Information and Systems, Vol. E101D, No. 5, 01.05.2018, p. 1296-1307.

Research output: Contribution to journalArticle

Yamaguchi, Masahiro ; Truong, Trong Phuc ; Mori, Shohei ; Nozick, Vincent ; Saito, Hideo ; Yachida, Shoji ; Sato, Hideaki. / Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry. In: IEICE Transactions on Information and Systems. 2018 ; Vol. E101D, No. 5. pp. 1296-1307.
@article{6c975da71c214b668f425c2c4599ef4b,
title = "Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry",
abstract = "In this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.",
keywords = "3D thermal map, Calibration, Delaunay division, Thermal-infrared camera, Visual odometry",
author = "Masahiro Yamaguchi and Truong, {Trong Phuc} and Shohei Mori and Vincent Nozick and Hideo Saito and Shoji Yachida and Hideaki Sato",
year = "2018",
month = "5",
day = "1",
doi = "10.1587/transinf.2017MVP0023",
language = "English",
volume = "E101D",
pages = "1296--1307",
journal = "IEICE Transactions on Information and Systems",
issn = "0916-8532",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "5",

}

TY - JOUR

T1 - Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry

AU - Yamaguchi, Masahiro

AU - Truong, Trong Phuc

AU - Mori, Shohei

AU - Nozick, Vincent

AU - Saito, Hideo

AU - Yachida, Shoji

AU - Sato, Hideaki

PY - 2018/5/1

Y1 - 2018/5/1

N2 - In this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

AB - In this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

KW - 3D thermal map

KW - Calibration

KW - Delaunay division

KW - Thermal-infrared camera

KW - Visual odometry

UR - http://www.scopus.com/inward/record.url?scp=85046290745&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046290745&partnerID=8YFLogxK

U2 - 10.1587/transinf.2017MVP0023

DO - 10.1587/transinf.2017MVP0023

M3 - Article

AN - SCOPUS:85046290745

VL - E101D

SP - 1296

EP - 1307

JO - IEICE Transactions on Information and Systems

JF - IEICE Transactions on Information and Systems

SN - 0916-8532

IS - 5

ER -