Outcome prediction from clinical text can prevent doctors from overlooking possible risks and help hospitals to plan capacities. Together with doctors from Charité Berlin, we developed use cases for Natural Language Processing in these scenarios. Our first focus are patients at admission time, when decision support can be especially valuable. We contribute a novel admission-to discharge task based on simulated patient admissions with four common outcome prediction targets: Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction.
The ideal system should infer outcomes based on symptoms, pre-conditions and risk factors of a patient. We evaluate the effectiveness of language models to handle this scenario and propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources. We further present a simple method to incorporate ICD code hierarchy into the models.
Our approach improves performance on the outcome tasks against several baselines. A detailed analysis reveals further strengths of the model, including transferability, but also weaknesses such as handling of vital values and inconsistencies in the underlying data.
In follow-up work, we analyze the behavior of our model regarding certain patient characteristics such as age, gender and ethnicity. We find that some learned patterns are medical plausible, while others don't follow medical rationale. We present a framework for behavioral testing of clinical NLP models that allows to communicate of such patterns to medical professionals.
Publications
Betty van Aken, Jens-Michalis Papaioannou, Manuel Mayrdorfer, Klemens Budde, Felix Gers, Alexander Löser. Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration. The 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL '21)
Paper: https://aclanthology.org/2021.eacl-main.75
Video: https://bit.ly/3SBZrMu
Demo: https://outcome-prediction.demo.datexis.com
Code: https://github.com/bvanaken/clinical-outcome-prediction
Model Checkpoint: https://huggingface.co/bvanaken/CORe-clinical-outcome-biobert-v1
Betty van Aken, Sebastian Herrmann, Alexander Löser. 2022. What Do You See in this Patient? Behavioral Testing of Clinical NLP Models. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 63–73, Seattle, WA. Association for Computational Linguistics.
Paper: https://aclanthology.org/2022.clinicalnlp-1.7
Code: https://github.com/bvanaken/clinical-assertion-data
As a next step, we direct our research on interpretable approaches to Clinical Outcome Prediction. We investigate how Prototypical Networks can present doctors with outcome results that are more useful than probability scores alone. This way, we want to further improve clinical decision support systems.
