Negation-based transfer learning for improving biomedical Named Entity Recognition and Relation Extraction.
Hermenegildo Fabregat, Andrés Duque, Juan Martínez-Romo, Lourdes Araujo
J. Biomedical Informatics 138: 104279 (2023)

Background and Objectives: Named Entity Recognition (NER) and Relation Extraction (RE) are two of the most
studied tasks in biomedical Natural Language Processing (NLP). The detection of specific terms
and entities and the relationships between them are key aspects for the development of more complex automatic
systems in the biomedical field. In this work, we explore transfer learning techniques for incorporating
information about negation into systems performing NER and RE. The main purpose of this research is to analyse
to what extent the successful detection of negated entities in separate tasks helps in the detection of
biomedical entities and their relationships.

Methods: Three neural architectures are proposed in this work, all of them mainly based on Bidirectional Long
Short-Term Memory (Bi-LSTM) networks and Conditional Random Fields (CRFs). While the first architecture is devoted
to detecting triggers and scopes of negated entities in any domain, two specific models are developed for
performing isolated NER tasks and joint NER and RE tasks in the biomedical domain. Then, weights related to
negation detection learned by the first architecture are incorporated into those last models. Two different
languages, Spanish and English, are taken into account in the experiments.

Results: Performance of the biomedical models is analysed both when the weights of the neural networks are
randomly initialized, and when weights from the negation detection model are incorporated into them. Improvements
of around 3.5% of F-Measure in the English language and more than 7% in the Spanish language are achieved in the
NER task, while the NER+RE task increases F-Measure scores by more than 13%
for the NER submodel and around 2% for the RE submodel.

Conclusions: The obtained results allow us to conclude that negation-based transfer learning techniques are
appropriate for performing biomedical NER and RE tasks. These results highlight the importance of detecting
negation for improving the identification of biomedical entities and their relationships. The explored techniques
show robustness by maintaining consistent results and improvements across different tasks and languages.