Bayesian 3D tracking from monocular video

Ernesto Brau, Jinyan Guan, Kyle Simek, Luca Del Pero, Colin Reimer Dawson, Jacobus J Barnard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

We develop a Bayesian modeling approach for tracking people in 3D from monocular video with unknown cameras. Modeling in 3D provides natural explanations for occlusions and smoothness discontinuities that result from projection, and allows priors on velocity and smoothness to be grounded in physical quantities: meters and seconds vs. pixels and frames. We pose the problem in the context of data association, in which observations are assigned to tracks. A correct application of Bayesian inference to multi-target tracking must address the fact that the model's dimension changes as tracks are added or removed, and thus, posterior densities of different hypotheses are not comparable. We address this by marginalizing out the trajectory parameters so the resulting posterior over data associations has constant dimension. This is made tractable by using (a) Gaussian process priors for smooth trajectories and (b) approximately Gaussian likelihood functions. Our approach provides a principled method for incorporating multiple sources of evidence, we present results using both optical flow and object detector outputs. Results are comparable to recent work on 3D tracking and, unlike others, our method requires no pre-calibrated cameras.

Original languageEnglish (US)
Title of host publicationProceedings of the IEEE International Conference on Computer Vision
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3368-3375
Number of pages8
ISBN (Print)9781479928392
DOIs
StatePublished - 2013
Event2013 14th IEEE International Conference on Computer Vision, ICCV 2013 - Sydney, NSW, Australia
Duration: Dec 1 2013Dec 8 2013

Other

Other2013 14th IEEE International Conference on Computer Vision, ICCV 2013
CountryAustralia
CitySydney, NSW
Period12/1/1312/8/13

Fingerprint

Cameras
Trajectories
Optical flows
Target tracking
Pixels
Detectors

Keywords

  • 3D scene modeling
  • Bayesian inference
  • MCMCDA
  • multi-object tracking

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Brau, E., Guan, J., Simek, K., Pero, L. D., Dawson, C. R., & Barnard, J. J. (2013). Bayesian 3D tracking from monocular video. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3368-3375). [6751530] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2013.418

Bayesian 3D tracking from monocular video. / Brau, Ernesto; Guan, Jinyan; Simek, Kyle; Pero, Luca Del; Dawson, Colin Reimer; Barnard, Jacobus J.

Proceedings of the IEEE International Conference on Computer Vision. Institute of Electrical and Electronics Engineers Inc., 2013. p. 3368-3375 6751530.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Brau, E, Guan, J, Simek, K, Pero, LD, Dawson, CR & Barnard, JJ 2013, Bayesian 3D tracking from monocular video. in Proceedings of the IEEE International Conference on Computer Vision., 6751530, Institute of Electrical and Electronics Engineers Inc., pp. 3368-3375, 2013 14th IEEE International Conference on Computer Vision, ICCV 2013, Sydney, NSW, Australia, 12/1/13. https://doi.org/10.1109/ICCV.2013.418
Brau E, Guan J, Simek K, Pero LD, Dawson CR, Barnard JJ. Bayesian 3D tracking from monocular video. In Proceedings of the IEEE International Conference on Computer Vision. Institute of Electrical and Electronics Engineers Inc. 2013. p. 3368-3375. 6751530 https://doi.org/10.1109/ICCV.2013.418
Brau, Ernesto ; Guan, Jinyan ; Simek, Kyle ; Pero, Luca Del ; Dawson, Colin Reimer ; Barnard, Jacobus J. / Bayesian 3D tracking from monocular video. Proceedings of the IEEE International Conference on Computer Vision. Institute of Electrical and Electronics Engineers Inc., 2013. pp. 3368-3375
@inproceedings{b685f61f451a4f6f8b6774df38fd10df,
title = "Bayesian 3D tracking from monocular video",
abstract = "We develop a Bayesian modeling approach for tracking people in 3D from monocular video with unknown cameras. Modeling in 3D provides natural explanations for occlusions and smoothness discontinuities that result from projection, and allows priors on velocity and smoothness to be grounded in physical quantities: meters and seconds vs. pixels and frames. We pose the problem in the context of data association, in which observations are assigned to tracks. A correct application of Bayesian inference to multi-target tracking must address the fact that the model's dimension changes as tracks are added or removed, and thus, posterior densities of different hypotheses are not comparable. We address this by marginalizing out the trajectory parameters so the resulting posterior over data associations has constant dimension. This is made tractable by using (a) Gaussian process priors for smooth trajectories and (b) approximately Gaussian likelihood functions. Our approach provides a principled method for incorporating multiple sources of evidence, we present results using both optical flow and object detector outputs. Results are comparable to recent work on 3D tracking and, unlike others, our method requires no pre-calibrated cameras.",
keywords = "3D scene modeling, Bayesian inference, MCMCDA, multi-object tracking",
author = "Ernesto Brau and Jinyan Guan and Kyle Simek and Pero, {Luca Del} and Dawson, {Colin Reimer} and Barnard, {Jacobus J}",
year = "2013",
doi = "10.1109/ICCV.2013.418",
language = "English (US)",
isbn = "9781479928392",
pages = "3368--3375",
booktitle = "Proceedings of the IEEE International Conference on Computer Vision",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Bayesian 3D tracking from monocular video

AU - Brau, Ernesto

AU - Guan, Jinyan

AU - Simek, Kyle

AU - Pero, Luca Del

AU - Dawson, Colin Reimer

AU - Barnard, Jacobus J

PY - 2013

Y1 - 2013

N2 - We develop a Bayesian modeling approach for tracking people in 3D from monocular video with unknown cameras. Modeling in 3D provides natural explanations for occlusions and smoothness discontinuities that result from projection, and allows priors on velocity and smoothness to be grounded in physical quantities: meters and seconds vs. pixels and frames. We pose the problem in the context of data association, in which observations are assigned to tracks. A correct application of Bayesian inference to multi-target tracking must address the fact that the model's dimension changes as tracks are added or removed, and thus, posterior densities of different hypotheses are not comparable. We address this by marginalizing out the trajectory parameters so the resulting posterior over data associations has constant dimension. This is made tractable by using (a) Gaussian process priors for smooth trajectories and (b) approximately Gaussian likelihood functions. Our approach provides a principled method for incorporating multiple sources of evidence, we present results using both optical flow and object detector outputs. Results are comparable to recent work on 3D tracking and, unlike others, our method requires no pre-calibrated cameras.

AB - We develop a Bayesian modeling approach for tracking people in 3D from monocular video with unknown cameras. Modeling in 3D provides natural explanations for occlusions and smoothness discontinuities that result from projection, and allows priors on velocity and smoothness to be grounded in physical quantities: meters and seconds vs. pixels and frames. We pose the problem in the context of data association, in which observations are assigned to tracks. A correct application of Bayesian inference to multi-target tracking must address the fact that the model's dimension changes as tracks are added or removed, and thus, posterior densities of different hypotheses are not comparable. We address this by marginalizing out the trajectory parameters so the resulting posterior over data associations has constant dimension. This is made tractable by using (a) Gaussian process priors for smooth trajectories and (b) approximately Gaussian likelihood functions. Our approach provides a principled method for incorporating multiple sources of evidence, we present results using both optical flow and object detector outputs. Results are comparable to recent work on 3D tracking and, unlike others, our method requires no pre-calibrated cameras.

KW - 3D scene modeling

KW - Bayesian inference

KW - MCMCDA

KW - multi-object tracking

UR - http://www.scopus.com/inward/record.url?scp=84898821643&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898821643&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2013.418

DO - 10.1109/ICCV.2013.418

M3 - Conference contribution

SN - 9781479928392

SP - 3368

EP - 3375

BT - Proceedings of the IEEE International Conference on Computer Vision

PB - Institute of Electrical and Electronics Engineers Inc.

ER -