Deep learning technology promotes fluorescence molecular tomography


Fluorescence imaging is widely used in biomedical research to study molecular and cellular processes in cell culture or tissue samples. This is motivated by the high inherent sensitivity of fluorescence techniques, the spatial resolution that compares favorably with cellular dimensions, the stability of the fluorescent labels used and the sophisticated labeling strategies that have been developed for selectively labeling target molecules. As such, this technique has emerged as a promising and noninvasive in vivo functional imaging modality.

As of now, it is well known that that the reason for fluorescence molecular tomography (FMT) reconstruction results with relatively low spatial resolution is the ill-posedness of the FMT inverse problem. For instance, in conventional methods, forward photon propagation modeling related to optical parameters need to be established, and the inverse problem needs to be solved afterwards. Recent publications have demonstrated that nonlinear models based on inverse problem simulation coupled with machine and deep learning, perform better than traditional analytical methods. Unfortunately, deep learning technology has not been applied to FMT three-dimensional (3D) reconstruction up to now.

To this end, developing a deep neural network (DNN) framework, a 3D deep encoder–decoder (3D-En–Decoder) network based on spatial convolution for FMT 3D reconstruction is essential. A group of Beihang University researchers led by Professor Guanglei Zhang in collaboration with Dr. Fei Liu at the Beijing Information Science proposed an end-to-end three-dimensional deep encoder–decoder (3D-En–Decoder) network. Their goal was to improve the quality of FMT reconstruction. Their work is currently published in the research journal, Optics Letters.

Unlike traditional methods in which the FMT reconstruction problem is explicitly defined and domain-knowledge is carefully engineered into the solution, their proposed network did not benefit from such prior knowledge but instead adopted large data sets to learn the unknown solution to the inverse problem. Ideally, the proposed 3D-En–Decoder, an end-to-end DNN, was designed to establish a nonlinear mapping from input to output.

The research team, using this novel deep learning-based approach, directly established the nonlinear mapping relationship between the inside fluorescent source distribution and the boundary fluorescent signal distribution. This observation pointed out that the reconstruction inaccuracy caused by the simplified linear model could be fundamentally avoided by the proposed network.

In summary, an end-to-end spatial CNN-based 3D-En–Decoder network was proposed to achieve high-accuracy 3D reconstruction. Overall, the reconstruction results of both simulation and phantom experiments proved that the 3D-En–Decoder network could achieve better reconstruction image contrast and localization accuracy for fluorescent targets comparing with conventional iteration-based regularized methods. Consequently, with this technique, any inaccuracy caused by establishing the photon propagation model or solving the ill-posed inverse problem can be fundamentally avoided, since the forward and inverse problems do not need to be explicitly solved anymore.

Deep learning technology promotes fluorescence molecular tomography - Advances in Engineering
Fig. 1. Model architecture of the 3D deep encoder-decoder (3D-En-Decoder) network for fluorescence molecular tomography (FMT).
Deep learning technology promotes fluorescence molecular tomography - Advances in Engineering
Fig. 2. Reconstruction results of FMT. The left parts were reconstructed with the conventional L1-regularized algorithm, while the right parts were reconstructed with the proposed 3D-En-Decoder network.

About the author

Guanglei Zhang is an associate professor in Beijing Advanced Innovation Center for Biomedical Engineering (BAICBME), Beihang University, China. In 2014, he received a doctor’s degree in Biomedical Engineering from Tsinghua University, China, and from 2015 to 2016, he conducted his postdoctoral research in Stanford University School of medicine, USA. From 2017 to 2018, he was an associate professor and deputy director of the Institute of Medical Intelligence, Beijing Jiaotong University. In 2018, he joined the BAICBME of Beihang University.

His primary research interests are focused on biomedical imaging, including optical molecule imaging and medical image analysis. Especially, he is now working on applying the cutting-edge artificial intelligence methods to the field of biomedical imaging.


Lin Guo, Fei Liu, Chuangjian Cai, Jie Liu, Guanglei Zhang. 3D deep encoder–decoder network for fluorescence molecular tomography. Volume 44, Number 8 / 2019 / Optics Letters.

Go To Optics Letters

Check Also

Towards anti-spoofing biometrics: 3D talking face - Advances in Engineering

Towards anti-spoofing biometrics: 3D talking face