Explainability for NLP in Pharmacovigilance: A Study on Adverse Event Report Triage in Swedish

Luise Dürlich, Erik Bergman, Maria Larsson, Hercules Dalianis, Seamus Doyle, Gabriel Westman, Joakim Nivre


Abstract
In fields like healthcare and pharmacovigilance, explainability has been raised as one way of approaching regulatory compliance with machine learning and automation.This paper explores two feature attribution methods to explain predictions of four different classifiers trained to assess the seriousness of adverse event reports. On a global level, differences between models and how well important features for serious predictions align with regulatory criteria for what constitutes serious adverse reactions are analysed. In addition, explanations of reports with incorrect predictions are manually explored to find systematic features explaining the misclassification.We find that while all models seemingly learn the importance of relevant concepts for adverse event report triage, the priority of these concepts varies from model to model and between explanation methods, and the analysis of misclassified reports indicates that reporting style may affect prediction outcomes.
Anthology ID:
2025.cl4health-1.5
Volume:
Proceedings of the Second Workshop on Patient-Oriented Language Processing (CL4Health)
Month:
May
Year:
2025
Address:
Albuquerque, New Mexico
Editors:
Sophia Ananiadou, Dina Demner-Fushman, Deepak Gupta, Paul Thompson
Venues:
CL4Health | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
46–68
Language:
URL:
https://aclanthology.org/2025.cl4health-1.5/
DOI:
Bibkey:
Cite (ACL):
Luise Dürlich, Erik Bergman, Maria Larsson, Hercules Dalianis, Seamus Doyle, Gabriel Westman, and Joakim Nivre. 2025. Explainability for NLP in Pharmacovigilance: A Study on Adverse Event Report Triage in Swedish. In Proceedings of the Second Workshop on Patient-Oriented Language Processing (CL4Health), pages 46–68, Albuquerque, New Mexico. Association for Computational Linguistics.
Cite (Informal):
Explainability for NLP in Pharmacovigilance: A Study on Adverse Event Report Triage in Swedish (Dürlich et al., CL4Health 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.cl4health-1.5.pdf

OSZAR »