Model-Heterogeneous Federated Learning for TinyML in IoT

The growing adoption of Internet of Things (IoT) technologies has led to a proliferation of smart devices capable of sensing and processing data locally. However, the limited computational, memory, and energy resources of such devices make the deployment of conventional deep learning models impractical. Tiny Machine Learning (TinyML) enables the execution of lightweight models directly on microcontrollers, but the heterogeneity of IoT devices naturally implies the use of different model sizes to maximize performance on each device. However, this introduces significant challenges for distributed learning frameworks such as Federated Learning (FL), which are typically employed in this context. This paper proposes a FL architecture specifically designed for TinyML on heterogeneous devices. The approach maximizes resource utilization by allowing each client to train a model whose size matches its hardware constraints: larger devices employ expanded models, while smaller ones use pruned or compact versions. A central server aggregates the heterogeneous models through either parameter averaging over shared substructures or knowledge distillation, enabling the construction of a unified global model without requiring identical architectures. Preliminary experiments on the MNIST dataset demonstrate that the proposed framework can effectively train and aggregate models of different sizes while maintaining competitive accuracy, thereby providing a practical solution for FL in resource-constrained IoT environments.