Natural language processing of prehospital emergency medical services trauma records allows for automated characterization of treatment appropriateness

BACKGROUND Incomplete prehospital trauma care is a significant contributor to preventable deaths. Current databases lack timelines easily constructible of clinical events. Temporal associations and procedural indications are critical to characterize treatment appropriateness. Natural language processing (NLP) methods present a novel approach to bridge this gap. We sought to evaluate the efficacy of a novel and automated NLP pipeline to determine treatment appropriateness from a sample of prehospital EMS motor vehicle crash records. METHODS A total of 142 records were used to extract airway procedures, intraosseous/intravenous access, packed red blood cell transfusion, crystalloid bolus, chest compression system, tranexamic acid bolus, and needle decompression. Reports were processed using four clinical NLP systems and augmented via a word2phrase method leveraging a large integrated health system clinical note repository to identify terms semantically similar with treatment indications. Indications were matched with treatments and categorized as indicated, missed (indicated but not performed), or nonindicated. Automated results were then compared with manual review, and precision and recall were calculated for each treatment determination. RESULTS Natural language processing identified 184 treatments. Automated timeline summarization was completed for all patients. Treatments were characterized as indicated in a subset of cases including the following: 69% (18 of 26 patients) for airway, 54.5% (6 of 11 patients) for intraosseous access, 11.1% (1 of 9 patients) for needle decompression, 55.6% (10 of 18 patients) for tranexamic acid, 60% (9 of 15 patients) for packed red blood cell, 12.9% (4 of 31 patients) for crystalloid bolus, and 60% (3 of 5 patients) for chest compression system. The most commonly nonindicated treatment was crystalloid bolus (22 of 142 patients). Overall, the automated NLP system performed with high precision and recall with over 70% of comparisons achieving precision and recall of greater than 80%. CONCLUSION Natural language processing methodologies show promise for enabling automated extraction of procedural indication data and timeline summarization. Future directions should focus on optimizing and expanding these techniques to scale and facilitate broader trauma care performance monitoring. LEVEL OF EVIDENCE Diagnostic tests or criteria, level III.