Robust spatiotemporal matching of electronic slides to presentation videos

Quanfu Fan, Kobus Barnard, Arnon Amir, Alon Efrat

Research output: Contribution to journalArticle

14 Scopus citations

Abstract

We describe a robust and efficient method for automatically matching and time-aligning electronic slides to videos of corresponding presentations. Matching electronic slides to videos provides new methods for indexing, searching, and browsing videos in distance-learning applications. However, robust automatic matching is challenging due to varied frame composition, slide distortion, camera movement, low-quality video capture, and arbitrary slides sequence. Our fully automatic approach combines image-based matching of slide to video frames with a temporal model for slide changes and camera events. To address these challenges, we begin by extracting scale-invariant feature-transformation (SIFT) keypoints from both slides and video frames, and matching them subject to a consistent projective transformation (homography) by using random sample consensus (RANSAC). We use the initial set of matches to construct a background model and a binary classifier for separating video frames showing slides from those without. We then introduce a new matching scheme for exploiting less distinctive SIFT keypoints that enables us to tackle more difficult images. Finally, we improve upon the matching based on visual information by using estimated matching probabilities as part of a hidden Markov model (HMM) that integrates temporal information and detected camera operations. Detailed quantitative experiments characterize each part of our approach and demonstrate an average accuracy of over 95% in 13 presentation videos.

Original languageEnglish (US)
Article number5705574
Pages (from-to)2315-2328
Number of pages14
JournalIEEE Transactions on Image Processing
Volume20
Issue number8
DOIs
StatePublished - Aug 1 2011

Keywords

  • Distance learning
  • homography constraint
  • matching slides to video frames
  • scale-invariant feature-transformation (SIFT) keypoints
  • video indexing and browsing

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Robust spatiotemporal matching of electronic slides to presentation videos'. Together they form a unique fingerprint.

  • Cite this