CADHE: Privacy-Preserving Medical Image Analysis Through Homomorphic Encrypted Convolutional Networks

Hospitals and clinical centres face significant barriers to data sharing due to stringent privacy regulations and security concerns, which hamper the collaborative development of robust AI diagnostic systems. In this scenario, Homomorphic Encryption (HE) emerges as a privacy-preserving approach to limit data exposure and enable the training of neural networks directly on encrypted data, making it ideal for medical imaging applications. However, the design of HE-based networks still remains highly challenging due to the complexity of operations on encrypted data and the lack of architectures specifically optimized for medical imaging tasks. This paper proposes CadHe,a lightweight Convolutional Neural Network architecture specifically designed to perform medical image analysis directly on homomorphically encrypted data. Our approach achieves end-to-end privacy preservation while maintaining diagnostic accuracy comparable to plaintext baselines. We demonstrate the effectiveness of Cadhe through extensive experimental evaluations on binary and multi-class classification tasks of the Expanded Disability Status Scale (EDSS) from multi-prospective Magnetic Resonance Imaging (MRI) images of multiple sclerosis patients. Cadhe achieved performance comparable to its unencrypted counterpart, reaching 88% accuracy on the all-modalities dataset and 83% in Tl for the binary task, and 70% and 82% accuracy on the same datasets for the multi-class task. These results confirm that Cadhe holds higher performances across settings, demonstrating the effectiveness of adapting convolutional operations to the encrypted domain.