Back to Publications
conference · 2024

Low-Resource Adaptation of Large Language Models for Regional Dialects

Abhiraj Bibhar, Aditi Singh, Yue Zhou · Conference on Empirical Methods in Natural Language Processing (EMNLP)

NLPLow-resource

Abstract

Large language models perform unevenly across regional dialects that are underrepresented in web-scale training corpora. We propose a parameter-efficient adaptation method that fine-tunes a small set of dialect-specific adapter layers using as few as 5,000 labeled examples, substantially closing the performance gap on downstream tasks including sentiment analysis, named entity recognition, and machine translation for four regional dialect groups.

Key contributions

  • A parameter-efficient adapter architecture requiring under 2% additional trainable parameters.
  • Evaluation across four dialect groups and three downstream tasks.
  • A released dataset of dialect-annotated examples to support future low-resource NLP research.
PDF DOI Publisher
Back to all publications

On this page

Authors

  • A Abhiraj Bibhar (this author)
  • A Aditi Singh
  • Y Yue Zhou

Cite this paper

Bibhar, A., Singh, A. & Zhou, Y. (2024). Low-Resource Adaptation of Large Language Models for Regional Dialects. Conference on Empirical Methods in Natural Language Processing (EMNLP).

@inproceedings{bibhar2024low,
  title     = {Low-Resource Adaptation of Large Language Models for Regional Dialects},
  author    = {Bibhar, Abhiraj and Singh, Aditi and Zhou, Yue},
  booktitle   = {Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2024}
}