Attention-Guided Federated Learning for Privacy-Preserving Clinical NLP
Abstract
Training clinical natural language processing models typically requires centralizing patient records, which is often legally or ethically infeasible across institutions. We introduce an attention-guided federated learning protocol that allows clinical language models to be trained collaboratively across multiple hospital systems without any patient data leaving its host institution.
Our method reduces communication overhead by approximately 40% relative to standard federated averaging by sharing only compressed attention summaries between rounds, rather than full model gradients. We provide a formal differential-privacy analysis with tight utility bounds and validate the approach across four real clinical NLP tasks — de-identification, readmission-risk prediction, adverse event detection, and discharge summarization — using data from four independent hospital systems.
Key contributions
- A lightweight attention-sharing protocol for cross-institution federated training.
- Formal differential-privacy guarantees with empirically validated utility bounds.
- Evaluation across four real-world clinical NLP tasks and four hospital systems.
- An open-source reference implementation released alongside the paper.
On this page
Authors
Cite this paper
Bibhar, A., Chen, L. & Okafor, T. (2026). Attention-Guided Federated Learning for Privacy-Preserving Clinical NLP. Journal of Machine Learning Research (JMLR).
@article{bibhar2026attention,
title = {Attention-Guided Federated Learning for Privacy-Preserving Clinical NLP},
author = {Bibhar, Abhiraj and Chen, Lin and Okafor, Tunde},
journal = {Journal of Machine Learning Research (JMLR)},
year = {2026}
}