Back to Publications
workshop · 2023

Toward Reproducible Benchmarks in Federated Learning Research

Abhiraj Bibhar · Workshop on Reproducibility in ML (NeurIPS Workshop)

Federated LearningReproducibility

Abstract

Federated learning papers frequently report results on incompatible experimental setups — differing client counts, non-IID partitioning strategies, and communication budgets — making cross-paper comparison unreliable. This position paper proposes a minimal standardized benchmark protocol and reports a re-evaluation of six widely cited federated learning methods under matched conditions, finding that reported performance gaps shrink substantially once experimental setup is controlled for.

Key contributions

  • A minimal standardized protocol for federated learning benchmark reporting.
  • A controlled re-evaluation of six prior methods showing reduced performance gaps under matched conditions.
  • Reference implementations and configuration files to support reproducible follow-up work.
PDF DOI Publisher
Back to all publications

On this page

Authors

  • A Abhiraj Bibhar (this author)

Cite this paper

Bibhar, A. (2023). Toward Reproducible Benchmarks in Federated Learning Research. Workshop on Reproducibility in ML (NeurIPS Workshop).

@inproceedings{bibhar2023toward,
  title     = {Toward Reproducible Benchmarks in Federated Learning Research},
  author    = {Bibhar, Abhiraj},
  booktitle   = {Workshop on Reproducibility in ML (NeurIPS Workshop)},
  year      = {2023}
}