Residual learning of video frame interpolation using convolutional LSTM

Keito Suzuki, Masaaki Ikehara

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

Video frame interpolation aims to generate intermediate frames between the original frames. This produces videos with a higher frame r ate and creates smoother motion. Many video frame interpolation methods first estimate the motion vector between the input frames and then synthesizes the intermediate frame based on the motion. However, these methods rely on the accuracy of the motion estimation step and fail to accurately generate the interpolated frame when the estimated motion vectors are inaccurate. Therefore, to avoid the uncertainties caused by motion estimation, this paper proposes a method that directly generates the intermediate frame. Since two consecutive frames are relatively similar, our method takes the average of these two frames and utilizes residual learning to learn the difference between the average of these frames and the ground truth middle frame. In addition, our method uses Convolutional LSTMs and four input frames to better incorporate spatiotemporal information. This neural network can be easily trained end to end without difficult to obtain data such as optical flow. Our experimental results show that the proposed method can perform favorably against other state-of-the-art frame interpolation methods.

本文言語English
ホスト出版物のタイトルProceedings of ICPR 2020 - 25th International Conference on Pattern Recognition
出版社Institute of Electrical and Electronics Engineers Inc.
ページ1499-1504
ページ数6
ISBN(電子版)9781728188089
DOI
出版ステータスPublished - 2020
イベント25th International Conference on Pattern Recognition, ICPR 2020 - Virtual, Milan, Italy
継続期間: 2021 1月 102021 1月 15

出版物シリーズ

名前Proceedings - International Conference on Pattern Recognition
ISSN(印刷版)1051-4651

Conference

Conference25th International Conference on Pattern Recognition, ICPR 2020
国/地域Italy
CityVirtual, Milan
Period21/1/1021/1/15

ASJC Scopus subject areas

  • コンピュータ ビジョンおよびパターン認識

フィンガープリント

「Residual learning of video frame interpolation using convolutional LSTM」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル