Dataset Culling: Towards Efficient Training of Distillation-Based Domain Specific Models

Kentaro Yoshioka, Edward Lee, Simon Wong, Mark Horowitz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Real-time CNN-based object detection models for applications like surveillance can achieve high accuracy but are computationally expensive. Recent works have shown 10 to 100× reduction in computation cost for inference by using domain-specific networks. However, prior works have focused on inference only. If the domain model requires frequent retraining, training costs can pose a significant bottleneck. To address this, we propose Dataset Culling: a pipeline to reduce the size of the dataset for training, based on the prediction difficulty. Images that are easy to classify are filtered out since they contribute little to improving the accuracy. The difficulty is measured using our proposed confidence loss metric with little computational overhead. Dataset Culling is extended to optimize the image resolution to further improve training and inference costs. We develop fixed-angle, long-duration video datasets across several domains, and we show that the dataset size can be culled by a factor of 300× to reduce the total training time by 47× with no accuracy loss or even with slight improvement.1

Original languageEnglish
Title of host publication2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
PublisherIEEE Computer Society
Pages3237-3241
Number of pages5
ISBN (Electronic)9781538662496
DOIs
Publication statusPublished - 2019 Sep
Externally publishedYes
Event26th IEEE International Conference on Image Processing, ICIP 2019 - Taipei, Taiwan, Province of China
Duration: 2019 Sep 222019 Sep 25

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2019-September
ISSN (Print)1522-4880

Conference

Conference26th IEEE International Conference on Image Processing, ICIP 2019
Country/TerritoryTaiwan, Province of China
CityTaipei
Period19/9/2219/9/25

Keywords

  • Dataset Culling
  • Deep Learning
  • Distillation
  • Object Detection
  • Training Efficiency

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint

Dive into the research topics of 'Dataset Culling: Towards Efficient Training of Distillation-Based Domain Specific Models'. Together they form a unique fingerprint.

Cite this