Deep Point Correspondence Estimation and SyN-Based Refinement for Multimodal Brain Image Registration

Document Type : Original Article

Authors

1 Department of Electrical and Electronic Engineering, Engineering Faculty, Shomal University, Amol, Iran

2 Department of Biomedical Engineering, Engineering Faculty, Shahed University, Tehran, Iran

Abstract

Accurate registration of preoperative Magnetic Resonance Imaging (MRI) with intraoperative ultrasound (US) is essential for effective neuronavigation, particularly in brain tumor surgeries where brain shift compromises anatomical fidelity. This study proposes a hybrid framework integrating a deep learning-based Multi-Layer Perceptron (MLP) with an optimization pipeline to enhance MR-to-US registration. The MLP is trained on paired anatomical landmarks extracted from the BITE and RESECT datasets to predict US coordinates from corresponding MRI points. An ensemble of five MLPs, weighted by inverse validation errors, is employed to estimate dense point correspondences, which are used to initialize an affine transformation. This transformation is refined using Symmetric Normalization (SyN) within the ANTs registration toolkit to model non-linear deformations. Quantitative evaluation demonstrates a mean squared error (MSE) of 0.1954 and a mean Euclidean distance of 4.97 mm—significantly outperforming a baseline rigid registration approach with 60% improvement in spatial alignment. The proposed pipeline executes in under 4 minutes per case on standard hardware, indicating potential for clinical integration. The results suggest combining learning-based correspondence prediction and classical registration yields accurate and computationally efficient multimodal Registration.

Keywords

References

  1. Velesaca, H. O., Bastidas, G., Rouhani, M., & Sappa, A. D. (2024). Multimodal image registration techniques: a comprehensive survey. Multimedia Tools and Applications, 83(23), 63919–63947. doi:10.1007/s11042-023-17991-2.
  2. Jena, R., Sethi, D., Chaudhari, P., & Gee, J. (2024). Deep learning in medical image registration: Magic or mirage?. 38th Conference on Neural Information Processing Systems (NeurIPS 2024), 37, 108331-108353, 9-15, December, 2024, Vancouver, Canada.
  3. Ahir, B. K., Engelhard, H. H., & Lakka, S. S. (2020). Tumor Development and Angiogenesis in Adult Brain Tumor: Glioblastoma. Molecular Neurobiology, 57(5), 2461–2478. doi:10.1007/s12035-020-01892-8.
  4. Jalalzadeh, A. H., Shalbaf, A., & Maghsoudi, A. (2021). Compensation of brain shift during surgery using non-rigid registration of MR and ultrasound images. Tehran University Medical Journal, 78(10), 658–667. http://tumj.tums.ac.ir/article-1-10930-en.html.
  5. Xiao, Y., Fortin, M., Unsgärd, G., Rivaz, H., & Reinertsen, I. (2017). REtroSpective Evaluation of Cerebral Tumors (RESECT): A clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries: A. Medical Physics, 44(7), 3875–3882. doi:10.1002/mp.12268.
  6. Avants, B. B., Epstein, C. L., Grossman, M., & Gee, J. C. (2008). Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 12(1), 26–41. doi:10.1016/j.media.2007.06.004.
  7. Mani, V. R. S., & Arivazhagan, S. (2013). Survey of medical image registration. Journal of Biomedical Engineering and Technology, 1(2), 8-25.
  8. Balasamy, K., Seethalakshmi, V., & Suganyadevi, S. (2024). Medical Image Analysis Through Deep Learning Techniques: A Comprehensive Survey. Wireless Personal Communications, 137(3), 1685–1714. doi:10.1007/s11277-024-11428-1.
  9. Chen, J., Liu, Y., Wei, S., Bian, Z., Subramanian, S., Carass, A., Prince, J. L., & Du, Y. (2025). A survey on deep learning in medical image registration: New technologies, uncertainty, evaluation metrics, and beyond. Medical Image Analysis, 100, 103385. doi:10.1016/j.media.2024.103385.
  10. Ramadan, H., El Bourakadi, D., Yahyaouy, A., & Tairi, H. (2024). Medical image registration in the era of Transformers: A recent review. Informatics in Medicine Unlocked, 49, 101540. doi:10.1016/j.imu.2024.101540.
  11. Tang, K., Li, Z., Tian, L., Wang, L., & Zhu, Y. (2020). ADMIR-Affine and deformable medical image registration for drug-addicted brain images. IEEE Access, 8, 70960–70968. doi:10.1109/ACCESS.2020.2986829.
  12. Mok, T.C.W., & Chung, A.C.S. (2020). Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science, vol 12263. Springer, Cham, Switzerland. doi:10.1007/978-3-030-59716-0_21.
  13. Kim, B., Han, I., & Ye, J.C. (2022). DiffuseMorph: Unsupervised Deformable Image Registration Using Diffusion Model. Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13691. Springer, Cham, Switzerland. doi:10.1007/978-3-031-19821-2_20.
  14. Mok, T. C. W., & Chung, A. C. S. (2020). Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr42600.2020.00470.
  15. Jia, X., Bartlett, J., Zhang, T., Lu, W., Qiu, Z., Duan, J. (2022). U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?. Machine Learning in Medical Imaging. MLMI 2022. Lecture Notes in Computer Science, vol 13583. Springer, Cham, Switzerland. doi:10.1007/978-3-031-21014-3_16.
  16. Pielawski, N., Wetzer, E., Öfverstedt, J., Lu, J., Wählby, C., Lindblad, J., & Sladoje, N. (2020). CoMIR: Contrastive multimodal image representation for registration. Advances in neural information processing systems, 33, 18433-18444.
  17. Jalalzadeh, A. H., Talebi, S. S., & Kamangar, M. H. (2024). Two-step registration of rigid and non-rigid MR-iUS for brain shift compensation using transfer learning. 2024 20th CSI International Symposium on Artificial Intelligence and Signal Processing, AISP 2024, 1–5,. doi:10.1109/AISP61396.2024.10475261.
  18. Dalca, A. V., Balakrishnan, G., Guttag, J., & Sabuncu, M. R. (2019). Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Medical Image Analysis, 57, 226–236. doi:10.1016/j.media.2019.07.006.
  19. Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2019). VoxelMorph: A Learning Framework for Deformable Medical Image Registration. IEEE Transactions on Medical Imaging, 38(8), 1788–1800. doi:10.1109/tmi.2019.2897538.
  20. Mercier, L., Del Maestro, R. F., Petrecca, K., Araujo, D., Haegelen, C., & Collins, D. L. (2012). Online database of clinical MR and ultrasound images of brain tumors. Medical Physics, 39(6), 3253–3261. doi:10.1118/1.4709600.
  21. Tustison, N. J., Yassa, M. A., Rizvi, B., Cook, P. A., Holbrook, A. J., Sathishkumar, M. T., Tustison, M. G., Gee, J. C., Stone, J. R., & Avants, B. B. (2024). ANTsX neuroimaging-derived structural phenotypes of UK Biobank. Scientific Reports, 14(1), 8848. doi:10.1038/s41598-024-59440-6.
  22. Tustison, N. J., Cook, P. A., Holbrook, A. J., Johnson, H. J., Muschelli, J., Devenyi, G. A., Duda, J. T., Das, S. R., Cullen, N. C., Gillen, D. L., Yassa, M. A., Stone, J. R., Gee, J. C., & Avants, B. B. (2021). The ANTsX ecosystem for quantitative biological and medical imaging. Scientific Reports, 11(1), 9068. doi:10.1038/s41598-021-87564-6.
  23. Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011). A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54(3), 2033–2044. doi:10.1016/j.neuroimage.2010.09.025.
  24. Razlighi, Q. R., Kehtarnavaz, N., & Yousefi, S. (2013). Evaluating similarity measures for brain image registration. Journal of Visual Communication and Image Representation, 24(7), 977–987. doi:10.1016/j.jvcir.2013.06.010.
Contributions of Science and Technology for Engineering
Volume 2, Issue 3
July 2025
Pages 18-25

Files

History

  • Receive Date: 28 April 2025
  • Revise Date: 20 May 2025
  • Accept Date: 20 May 2025
  • First Publish Date: 20 May 2025
  • Publish Date: 08 July 2025

Share

How to cite

Statistics