Constant velocity 3D convolution

Yusuke Sekikawa, Kohta Ishikawa, Kosuke Hara, Yuuichi Yoshida, Koichiro Suzuki, Ikuro Sato, Hideo Saito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We propose a novel three-dimensional (3D)-convolution method, cv3dconv, for detecting spatiotemporal features from videos. It reduces the number of sum-of-products of 3D convolution by thousands of times by assuming the constant moving velocity of the camera. We observed that a specific class of video sequences, such as those captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2D features in the temporal dimension. Our principal finding is that the 3D kernel, represented by the constant-velocity, can be decomposed into a convolution of a 2D kernel representing the shapes and a 3D kernel representing the velocity. We derived the efficient recursive algorithm for this class of 3D convolution which is exceptionally suited for sparse data, and this parameterized decomposed representation imposes a structured regularization along the temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of cv3dconv for the visual odometry estimation task using real event camera data captured in urban road scene.

Original languageEnglish
Title of host publicationProceedings - 2018 International Conference on 3D Vision, 3DV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages343-351
Number of pages9
ISBN (Electronic)9781538684252
DOIs
Publication statusPublished - 2018 Oct 12
Event6th International Conference on 3D Vision, 3DV 2018 - Verona, Italy
Duration: 2018 Sep 52018 Sep 8

Other

Other6th International Conference on 3D Vision, 3DV 2018
CountryItaly
CityVerona
Period18/9/518/9/8

Fingerprint

Convolution
Cameras

Keywords

  • 3D convolution
  • Constant velocity
  • Fourier transform

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Sekikawa, Y., Ishikawa, K., Hara, K., Yoshida, Y., Suzuki, K., Sato, I., & Saito, H. (2018). Constant velocity 3D convolution. In Proceedings - 2018 International Conference on 3D Vision, 3DV 2018 (pp. 343-351). [8490985] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/3DV.2018.00047

Constant velocity 3D convolution. / Sekikawa, Yusuke; Ishikawa, Kohta; Hara, Kosuke; Yoshida, Yuuichi; Suzuki, Koichiro; Sato, Ikuro; Saito, Hideo.

Proceedings - 2018 International Conference on 3D Vision, 3DV 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 343-351 8490985.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sekikawa, Y, Ishikawa, K, Hara, K, Yoshida, Y, Suzuki, K, Sato, I & Saito, H 2018, Constant velocity 3D convolution. in Proceedings - 2018 International Conference on 3D Vision, 3DV 2018., 8490985, Institute of Electrical and Electronics Engineers Inc., pp. 343-351, 6th International Conference on 3D Vision, 3DV 2018, Verona, Italy, 18/9/5. https://doi.org/10.1109/3DV.2018.00047
Sekikawa Y, Ishikawa K, Hara K, Yoshida Y, Suzuki K, Sato I et al. Constant velocity 3D convolution. In Proceedings - 2018 International Conference on 3D Vision, 3DV 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 343-351. 8490985 https://doi.org/10.1109/3DV.2018.00047
Sekikawa, Yusuke ; Ishikawa, Kohta ; Hara, Kosuke ; Yoshida, Yuuichi ; Suzuki, Koichiro ; Sato, Ikuro ; Saito, Hideo. / Constant velocity 3D convolution. Proceedings - 2018 International Conference on 3D Vision, 3DV 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 343-351
@inproceedings{11b5b248f4ea4dcdb181317f75c0edbd,
title = "Constant velocity 3D convolution",
abstract = "We propose a novel three-dimensional (3D)-convolution method, cv3dconv, for detecting spatiotemporal features from videos. It reduces the number of sum-of-products of 3D convolution by thousands of times by assuming the constant moving velocity of the camera. We observed that a specific class of video sequences, such as those captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2D features in the temporal dimension. Our principal finding is that the 3D kernel, represented by the constant-velocity, can be decomposed into a convolution of a 2D kernel representing the shapes and a 3D kernel representing the velocity. We derived the efficient recursive algorithm for this class of 3D convolution which is exceptionally suited for sparse data, and this parameterized decomposed representation imposes a structured regularization along the temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of cv3dconv for the visual odometry estimation task using real event camera data captured in urban road scene.",
keywords = "3D convolution, Constant velocity, Fourier transform",
author = "Yusuke Sekikawa and Kohta Ishikawa and Kosuke Hara and Yuuichi Yoshida and Koichiro Suzuki and Ikuro Sato and Hideo Saito",
year = "2018",
month = "10",
day = "12",
doi = "10.1109/3DV.2018.00047",
language = "English",
pages = "343--351",
booktitle = "Proceedings - 2018 International Conference on 3D Vision, 3DV 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Constant velocity 3D convolution

AU - Sekikawa, Yusuke

AU - Ishikawa, Kohta

AU - Hara, Kosuke

AU - Yoshida, Yuuichi

AU - Suzuki, Koichiro

AU - Sato, Ikuro

AU - Saito, Hideo

PY - 2018/10/12

Y1 - 2018/10/12

N2 - We propose a novel three-dimensional (3D)-convolution method, cv3dconv, for detecting spatiotemporal features from videos. It reduces the number of sum-of-products of 3D convolution by thousands of times by assuming the constant moving velocity of the camera. We observed that a specific class of video sequences, such as those captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2D features in the temporal dimension. Our principal finding is that the 3D kernel, represented by the constant-velocity, can be decomposed into a convolution of a 2D kernel representing the shapes and a 3D kernel representing the velocity. We derived the efficient recursive algorithm for this class of 3D convolution which is exceptionally suited for sparse data, and this parameterized decomposed representation imposes a structured regularization along the temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of cv3dconv for the visual odometry estimation task using real event camera data captured in urban road scene.

AB - We propose a novel three-dimensional (3D)-convolution method, cv3dconv, for detecting spatiotemporal features from videos. It reduces the number of sum-of-products of 3D convolution by thousands of times by assuming the constant moving velocity of the camera. We observed that a specific class of video sequences, such as those captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2D features in the temporal dimension. Our principal finding is that the 3D kernel, represented by the constant-velocity, can be decomposed into a convolution of a 2D kernel representing the shapes and a 3D kernel representing the velocity. We derived the efficient recursive algorithm for this class of 3D convolution which is exceptionally suited for sparse data, and this parameterized decomposed representation imposes a structured regularization along the temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of cv3dconv for the visual odometry estimation task using real event camera data captured in urban road scene.

KW - 3D convolution

KW - Constant velocity

KW - Fourier transform

UR - http://www.scopus.com/inward/record.url?scp=85056769213&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056769213&partnerID=8YFLogxK

U2 - 10.1109/3DV.2018.00047

DO - 10.1109/3DV.2018.00047

M3 - Conference contribution

AN - SCOPUS:85056769213

SP - 343

EP - 351

BT - Proceedings - 2018 International Conference on 3D Vision, 3DV 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -