When Heterogeneous Modalities Synchronize: Mitigating Desynchronization Attacks in Multimodal Online Federated Learning

Multimodal online federated learning (MOFL) relies on accurate temporal alignment across heterogeneous data streams to enable effective cross-modal fusion. In this paper, we investigate temporal desynchronization as a practical adversarial attack surface in such systems. We show that even small temporal shifts between modalities can disrupt cross-modal alignment, leading to degraded model performance and increased attack effectiveness. Unlike traditional poisoning attacks, the proposed approach operates at the synchronization level, making it subtle and difficult to detect. We conduct experiments under both untargeted and label-conditioned (targeted) attack settings to comprehensively evaluate the impact of temporal misalignment. To mitigate this vulnerability, we propose a lightweight Cross-Client Consistency Scoring Mechanism (CCCSM) that reweights client updates based on their similarity during server-side aggregation. Experimental results on the UP-Fall dataset demonstrate that (i) the impact of the attack increases with the magnitude of temporal misalignment and (ii) the attack is more severe under targeted settings. For perturbation strength k=5, the overall F1-score drops from 95.6% to 80.5%, while the attack success rate (ASR) increases up to 79.2% in targeted settings. The proposed CCCSM-based defense reduces ASR from 79.2% to 45.1% and improves overall F1 from 79.1% to 84.7% under targeted attacks with k=5, while maintaining performance close to the clean baseline. These findings highlight the importance of synchronization in MOFL and the need for synchronization-aware defenses in real-world deployments.