Summary
A novel machine learning model—composed of an ensemble of proprietary algorithms—that helps evaluate and recategorize miscellaneous patient safety events from a report full of free text narratives and unstructured language.
Inventors
Allan Fong
Raj Ratwani, PhD
Katherine Adams
Jessica Howe
What is it? What does it do?
Electronic patient safety reporting systems have attempted to improve efficiencies beyond traditional paper-based reporting systems; however, these systems usually contain incomplete and inaccurate reports that negatively affect their utility for medical error research. One of the most common types of patient safety event (PSE) reports documented in these systems is labeled “miscellaneous” which requires a user to perform an in-depth review to recategorize the event. While there have been advances in natural language processing (NLP) and machine learning, none have been applied to the recategorization of miscellaneous events, partly due to the difficulties of evaluation.
The novel machine learning model uses NLP to re-categorize miscellaneous-classified events into more discrete “clinically relevant categories;” determine the sentiment of the event (positive, negative, or neutral); and potentially report on the causal factors of the events. The model is fully integrated into a clinical workflow dashboard to enhance usability and visualization of the PSE reports.
Why is it better?
-
Improved accuracy of PSE report classification
-
Reduced manual recategorization, saving time and money
-
Seamless integration into clinical workflow dashboard that improves usability and interoperability
What is its current status?
The model was developed, trained, and tested using more than 60,000 PSE reports from a mid-Atlantic multi-hospital healthcare system from 2017-2019. Nine participants were recruited from the healthcare system to evaluate the clinical workflow dashboard. When compared to other standalone models, the PSE recategorization model performed the best on accuracy (0.92), precision (0.90), and recall (0.89) for reclassifying miscellaneous events. Participants using the clinical dashboard found it helpful and agreed that many of the miscellaneous reports in the PSE system could be reclassified.
MedStar Inventor Services performed a patentability and market analysis assessment on the invention that included a detailed report on the clinical need and competitive landscape. The team subsequently filed a utility patent that is currently pending.
The MedStar Inventor Services team is now seeking a licensing/collaboration partner to help advance and commercialize this technology. Please contact us at invent@medstar.net.
Related invention:
Publications
-
Fong et al. (2021). A Machine Learning Approach to Reclassifying Miscellaneous Patient Safety Event Reports. Journal of patient safety, 17(8), e829–e833.
https://pubmed.ncbi.nlm.nih.gov/32555052/
Case Studies
-
MedStar Health, “Improving Patient Safety Through Visual Analytics and Data Science”