Combining NLP with evidence-based methods to find text metrics related to perceived and actual text difficulty

Gondy Leroy, James E. Endicott

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

Measuring text difficulty is prevalent in health informatics since it is useful for information personalization and optimization. Unfortunately, it is uncertain how best to compute difficulty so that it relates to reader understanding. We aim to create computational, evidence-based metrics of perceived and actual text difficulty. We start with a corpus analysis to identify candidate metrics which are further tested in user studies. Our corpus contains blogs and journal articles (N=1,073) representing easy and difficult text. Using natural language processing, we calculated base grammatical and semantic metrics, constructed new composite metrics (noun phrase complexity and semantic familiarity), and measured the commonly used Flesch-Kincaid grade level. The metrics differed significantly between document types. Nouns were more prevalent but less familiar in difficult text; verbs and function words were more prevalent in easy text. Noun phrase complexity was lower, semantic familiarity was higher and grade levels were lower in easy text. Then, all metrics were tested for their relation to perceived and actual difficulty using follow-up analyses of two user studies conducted earlier. Base metrics and noun phrase complexity correlated significantly with perceived difficulty and could help explain actual difficulty.

Original languageEnglish (US)
Title of host publicationIHI'12 - Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium
Pages749-753
Number of pages5
DOIs
StatePublished - Mar 8 2012
Externally publishedYes
Event2nd ACM SIGHIT International Health Informatics Symposium, IHI'12 - Miami, FL, United States
Duration: Jan 28 2012Jan 30 2012

Publication series

NameIHI'12 - Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium

Other

Other2nd ACM SIGHIT International Health Informatics Symposium, IHI'12
CountryUnited States
CityMiami, FL
Period1/28/121/30/12

Keywords

  • Actual difficulty
  • Health informatics
  • Natural language processing
  • Perceived difficulty
  • Readability

ASJC Scopus subject areas

  • Health Informatics
  • Health Information Management

Fingerprint Dive into the research topics of 'Combining NLP with evidence-based methods to find text metrics related to perceived and actual text difficulty'. Together they form a unique fingerprint.

  • Cite this

    Leroy, G., & Endicott, J. E. (2012). Combining NLP with evidence-based methods to find text metrics related to perceived and actual text difficulty. In IHI'12 - Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium (pp. 749-753). (IHI'12 - Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium). https://doi.org/10.1145/2110363.2110452