Text simplification tools: Using machine learning to discover features that identify difficult text

David Kauchak, Obay Mouradi, Christopher Pentoney, Gondy Leroy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

Although providing understandable information is a critical component in healthcare, few tools exist to help clinicians identify difficult sections in text. We systematically examine sixteen features for predicting the difficulty of health texts using six different machine learning algorithms. Three represent new features not previously examined: medical concept density; specificity (calculated using word-level depth in MeSH); and ambiguity (calculated using the number of UMLS Metathesaurus concepts associated with a word). We examine these features for a binary prediction task on 118,000 simple and difficult sentences from a sentence-aligned corpus. Using all features, random forests is the most accurate with 84% accuracy. Model analysis of the six models and a complementary ablation study shows that the specificity and ambiguity features are the strongest predictors (24% combined impact on accuracy). Notably, a training size study showed that even with a 1% sample (1,062 sentences) an accuracy of 80% can be achieved.

Original languageEnglish (US)
Title of host publicationProceedings of the 47th Annual Hawaii International Conference on System Sciences, HICSS 2014
PublisherIEEE Computer Society
Pages2616-2625
Number of pages10
ISBN (Print)9781479925049
DOIs
StatePublished - Jan 1 2014
Event47th Hawaii International Conference on System Sciences, HICSS 2014 - Waikoloa, HI, United States
Duration: Jan 6 2014Jan 9 2014

Publication series

NameProceedings of the Annual Hawaii International Conference on System Sciences
ISSN (Print)1530-1605

Other

Other47th Hawaii International Conference on System Sciences, HICSS 2014
CountryUnited States
CityWaikoloa, HI
Period1/6/141/9/14

    Fingerprint

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Kauchak, D., Mouradi, O., Pentoney, C., & Leroy, G. (2014). Text simplification tools: Using machine learning to discover features that identify difficult text. In Proceedings of the 47th Annual Hawaii International Conference on System Sciences, HICSS 2014 (pp. 2616-2625). [6758930] (Proceedings of the Annual Hawaii International Conference on System Sciences). IEEE Computer Society. https://doi.org/10.1109/HICSS.2014.330