Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We propose a robust method for estimating dynamic 3D curvilinear branching structure from monocular images. While 3D reconstruction from images has been widely studied, estimating thin structure has received less attention. This problem becomes more challenging in the presence of camera error, scene motion, and a constraint that curves are attached in a branching structure. We propose a new generalpurpose prior, a branching Gaussian processes (BGP), that models spatial smoothness and temporal dynamics of curves while enforcing attachment between them. We apply this prior to fit 3D trees directly to image data, using an efficient scheme for approximate inference based on expectation propagation. The BGP prior’s Gaussian form allows us to approximately marginalize over 3D trees with a given model structure, enabling principled comparison between tree models with varying complexity. We test our approach on a novel multi-view dataset depicting plants with known 3D structures and topologies undergoing small nonrigid motion. Our method outperforms a state-of-the-art 3D reconstruction method designed for non-moving thin structure. We evaluate under several common measures, and we propose a new measure for reconstructions of branching multi-part 3D scenes under motion.

Original languageEnglish (US)
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
PublisherSpringer Verlag
Pages177-193
Number of pages17
Volume9912 LNCS
ISBN (Print)9783319464831
DOIs
StatePublished - 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9912 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Fingerprint

Branching process
Gaussian Process
Model structures
Branching
Cameras
3D Reconstruction
Topology
Motion
Curve
Robust Methods
Process Model
Smoothness
Camera
Propagation
Evaluate
Model

Keywords

  • Expectation propagation
  • Multiview stereo
  • Nonrigid models

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Simek, K., Palanivelu, R., & Barnard, J. J. (2016). Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees. In Computer Vision - 14th European Conference, ECCV 2016, Proceedings (Vol. 9912 LNCS, pp. 177-193). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46484-8_11

Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees. / Simek, Kyle; Palanivelu, Ravishankar; Barnard, Jacobus J.

Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Vol. 9912 LNCS Springer Verlag, 2016. p. 177-193 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Simek, K, Palanivelu, R & Barnard, JJ 2016, Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees. in Computer Vision - 14th European Conference, ECCV 2016, Proceedings. vol. 9912 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9912 LNCS, Springer Verlag, pp. 177-193. https://doi.org/10.1007/978-3-319-46484-8_11
Simek K, Palanivelu R, Barnard JJ. Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees. In Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Vol. 9912 LNCS. Springer Verlag. 2016. p. 177-193. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46484-8_11
Simek, Kyle ; Palanivelu, Ravishankar ; Barnard, Jacobus J. / Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Vol. 9912 LNCS Springer Verlag, 2016. pp. 177-193 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{5baf0c76dec2472e95a78ef78729c899,
title = "Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees",
abstract = "We propose a robust method for estimating dynamic 3D curvilinear branching structure from monocular images. While 3D reconstruction from images has been widely studied, estimating thin structure has received less attention. This problem becomes more challenging in the presence of camera error, scene motion, and a constraint that curves are attached in a branching structure. We propose a new generalpurpose prior, a branching Gaussian processes (BGP), that models spatial smoothness and temporal dynamics of curves while enforcing attachment between them. We apply this prior to fit 3D trees directly to image data, using an efficient scheme for approximate inference based on expectation propagation. The BGP prior’s Gaussian form allows us to approximately marginalize over 3D trees with a given model structure, enabling principled comparison between tree models with varying complexity. We test our approach on a novel multi-view dataset depicting plants with known 3D structures and topologies undergoing small nonrigid motion. Our method outperforms a state-of-the-art 3D reconstruction method designed for non-moving thin structure. We evaluate under several common measures, and we propose a new measure for reconstructions of branching multi-part 3D scenes under motion.",
keywords = "Expectation propagation, Multiview stereo, Nonrigid models",
author = "Kyle Simek and Ravishankar Palanivelu and Barnard, {Jacobus J}",
year = "2016",
doi = "10.1007/978-3-319-46484-8_11",
language = "English (US)",
isbn = "9783319464831",
volume = "9912 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "177--193",
booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",
address = "Germany",

}

TY - GEN

T1 - Branching gaussian processes with applications to spatiotemporal reconstruction of 3D trees

AU - Simek, Kyle

AU - Palanivelu, Ravishankar

AU - Barnard, Jacobus J

PY - 2016

Y1 - 2016

N2 - We propose a robust method for estimating dynamic 3D curvilinear branching structure from monocular images. While 3D reconstruction from images has been widely studied, estimating thin structure has received less attention. This problem becomes more challenging in the presence of camera error, scene motion, and a constraint that curves are attached in a branching structure. We propose a new generalpurpose prior, a branching Gaussian processes (BGP), that models spatial smoothness and temporal dynamics of curves while enforcing attachment between them. We apply this prior to fit 3D trees directly to image data, using an efficient scheme for approximate inference based on expectation propagation. The BGP prior’s Gaussian form allows us to approximately marginalize over 3D trees with a given model structure, enabling principled comparison between tree models with varying complexity. We test our approach on a novel multi-view dataset depicting plants with known 3D structures and topologies undergoing small nonrigid motion. Our method outperforms a state-of-the-art 3D reconstruction method designed for non-moving thin structure. We evaluate under several common measures, and we propose a new measure for reconstructions of branching multi-part 3D scenes under motion.

AB - We propose a robust method for estimating dynamic 3D curvilinear branching structure from monocular images. While 3D reconstruction from images has been widely studied, estimating thin structure has received less attention. This problem becomes more challenging in the presence of camera error, scene motion, and a constraint that curves are attached in a branching structure. We propose a new generalpurpose prior, a branching Gaussian processes (BGP), that models spatial smoothness and temporal dynamics of curves while enforcing attachment between them. We apply this prior to fit 3D trees directly to image data, using an efficient scheme for approximate inference based on expectation propagation. The BGP prior’s Gaussian form allows us to approximately marginalize over 3D trees with a given model structure, enabling principled comparison between tree models with varying complexity. We test our approach on a novel multi-view dataset depicting plants with known 3D structures and topologies undergoing small nonrigid motion. Our method outperforms a state-of-the-art 3D reconstruction method designed for non-moving thin structure. We evaluate under several common measures, and we propose a new measure for reconstructions of branching multi-part 3D scenes under motion.

KW - Expectation propagation

KW - Multiview stereo

KW - Nonrigid models

UR - http://www.scopus.com/inward/record.url?scp=84990020254&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990020254&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46484-8_11

DO - 10.1007/978-3-319-46484-8_11

M3 - Conference contribution

AN - SCOPUS:84990020254

SN - 9783319464831

VL - 9912 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 177

EP - 193

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

PB - Springer Verlag

ER -