Why Federated Learning Is Harder Than the Tutorials Make It Look

Every year I watch a fresh cohort of students implement FedAvg in an afternoon, get a clean accuracy curve on MNIST, and conclude that federated learning is basically solved. It isn’t. What tutorials skip is everything that makes federated learning federated in the first place: the data isn’t yours, the clients aren’t reliable, and the network is not a courtesy.

The four problems tutorials don’t show you

1. Non-IID data breaks your assumptions quietly

Most demos partition a single dataset randomly across simulated clients, so every client sees a roughly similar distribution. Real institutions don’t. A rural clinic’s patient population looks nothing like an urban teaching hospital’s. When client data distributions diverge sharply, naive averaging can actively hurt the global model relative to what any single well-resourced client could achieve alone.

2. Stragglers and dropouts are the norm, not the exception

In a simulation, every client always responds within one round. In practice, a meaningful fraction of clients drop out mid-round, respond late, or return corrupted updates due to hardware failures. Your aggregation strategy has to tolerate partial participation gracefully, or your global model will thrash between rounds.

3. Communication cost dominates, not compute

Tutorials optimize for accuracy per epoch. Deployments optimize for accuracy per megabyte transmitted, because clients are often on metered or unreliable connections. This changes which model architectures and compression strategies are actually viable.

4. Privacy guarantees require care, not vibes

“We never centralize the data” is not a privacy guarantee — gradient updates can leak surprising amounts of information about individual training examples. If you actually need a formal guarantee, you need differential privacy accounting done properly, and that accounting interacts with everything above: client sampling, number of rounds, and clipping strategy all affect your privacy budget.

What actually works

In our own deployments, three practices have mattered more than any particular aggregation algorithm:

1. Stratified client evaluation. Never trust a single global accuracy number. Track per-client performance distributions, especially for the worst-performing clients.
2. Asynchronous-tolerant aggregation. Design for partial participation from day one rather than bolting it on later.
3. Conservative privacy budgets with real accounting. Decide on your privacy target before you pick your architecture, not after.

None of this is exotic research — it’s mostly engineering discipline that tutorials, by their nature, don’t have room for. If you’re moving from a notebook to a real deployment, budget real time for exactly these four problems.