CharaParser+EQ: Performance evaluation without gold standard

Hong Cui, Wasila Dahdul, Alexander T. Dececchi, Nizar Ibrahim, Paula Mabee, James P. Balhoff, Hariharan Gopalakrishnan

Research output: Contribution to journalArticle

10 Scopus citations

Abstract

To make phenotypic characters of organisms widely useful for computerized biology research, biocurators manually convert character descriptions to a structured format, for example the Entity-Quality (EQ) format. The manual approach is time consuming and affected by inter-curator variations. In this paper we report a software application, CharaParser+EQ, to our knowledge the first software that produces EQ statements from textual character descriptions. We report a recent experiment that evaluates the performance of the software against three experienced biocurators. While the software is still far from being able to compete with biocurators on this highly intellectual task, the results show (1) CharaParser+EQ's performance (precision and recall) is greatly improved compared to a previous version, (2) the completeness of the ontologies used in the process has significant impact both on the software's EQ generation performance and on the agreement among curators, and (3) unlimited access to external knowledge (published papers, books) by curators has no significant impact on inter-curator agreements. A detailed error analysis that compares machine and curator generated EQs is included.

Original languageEnglish (US)
Pages (from-to)1-10
Number of pages10
JournalProceedings of the Association for Information Science and Technology
Volume52
Issue number1
DOIs
StatePublished - 2015

Keywords

  • EQ statements
  • Natural Language Processing
  • Phenotype character curation
  • curation inconsistency
  • ontology search

ASJC Scopus subject areas

  • Computer Science(all)
  • Library and Information Sciences

Fingerprint Dive into the research topics of 'CharaParser+EQ: Performance evaluation without gold standard'. Together they form a unique fingerprint.

  • Cite this