A gossip-based system for fast approximate score computation in multinomial Bayesian networks

Arun Zachariah, Praveen Rao, Anas Katib, Monica Senapati, Jacobus J Barnard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we present a system for fast approximate score computation, a fundamental task for score-based structure learning of multinomial Bayesian networks. Our work is motivated by the fact that exact score computation on large datasets is very time consuming. Our system enables approximate score computation on large datasets in an efficient and scalable manner with probabilistic error bounds on the statistics required for score computation. Our system has several novel features including gossip-based decentralized computation of statistics, lower resource consumption via a probabilistic approach of maintaining statistics, and effective distribution of tasks for score computation using hashing techniques. The demo will provide a real-time and interactive experience to a user on how our system employs the principle of gossiping and hashing techniques in a novel way for fast approximate score computation. The user will be able to control different aspects of our system's execution on a cluster with up to 32 nodes. The approximate scores output by our system can be then used by existing score-based structure learning algorithms.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE 35th International Conference on Data Engineering, ICDE 2019
PublisherIEEE Computer Society
Pages1968-1971
Number of pages4
ISBN (Electronic)9781538674741
DOIs
StatePublished - Apr 1 2019
Event35th IEEE International Conference on Data Engineering, ICDE 2019 - Macau, China
Duration: Apr 8 2019Apr 11 2019

Publication series

NameProceedings - International Conference on Data Engineering
Volume2019-April
ISSN (Print)1084-4627

Conference

Conference35th IEEE International Conference on Data Engineering, ICDE 2019
CountryChina
CityMacau
Period4/8/194/11/19

Keywords

  • Approximate score computation
  • Bayesian networks
  • Gossip algorithms
  • Large scale data

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Fingerprint Dive into the research topics of 'A gossip-based system for fast approximate score computation in multinomial Bayesian networks'. Together they form a unique fingerprint.

  • Cite this

    Zachariah, A., Rao, P., Katib, A., Senapati, M., & Barnard, J. J. (2019). A gossip-based system for fast approximate score computation in multinomial Bayesian networks. In Proceedings - 2019 IEEE 35th International Conference on Data Engineering, ICDE 2019 (pp. 1968-1971). [8731481] (Proceedings - International Conference on Data Engineering; Vol. 2019-April). IEEE Computer Society. https://doi.org/10.1109/ICDE.2019.00216