Cross-domain One-shot Video Object Detection

Abstract

One-shot-object detection (OSOD) aims to detect novel object classes using a single example of an unseen class. Cross-domain OSOD is a more challenging problem since the seen and unseen objects are sampled from the entirely disjoint datasets. The majority of the existing CD-OSOD methods focus on image datasets where the video domain remains largely unaddressed. To tackle this problem, we introduce a one-shot cross-domain video object detection (CD-OSVOD) model enabling adaptation from the still image to the video. Specifically the novel target object is designated as the query shot and a target driven cross-domain finetuning (FT) scheme is integrated with a baseline object detector. To address the requirements of the long term video object detection, the FT scheme is augmented with a novel Online Target Update (OTU) mechanism, enabling the detector to handle challenges such as appearance changes and occlusions. The OTU is controlled by a temporal aggregation module (TAM) which leverages temporal information in video and triggers update of the one-shot query when the temporal consistency is disrupted. The proposed CD-OSVOD utilizes base models trained on COCO and VOC still image datasets and successfully adapts to the video domain for novel object classes. Performance evaluations on challenging VOT-LT benchmarking video dataset demonstrate significant improvement in AP50 and mAP scores, highlighting the effectiveness of the proposed domain adaptation approach.