Novel Photonic Neural Networks Based on the Dynamic Model

Artificial intelligence (AI) has made incredible progress in transforming every walk of life, especially in the development of deep neural networks for the execution of scientific and industrial applications.  However, with the limitations of electronic transistors approaching a physical threshold, there is a growing interest in exploring alternative computing modalities. One promising avenue is photonic computing, where computations are performed using photons instead of electrons, offering the advantage of high-speed processing with low energy consumption.

The development of photonic computing and its applications. Traditional computing processors, such as field programmable gate arrays, application-specific integrated circuits, and graphics or tensor processing units, have been extensively used for implementing deep neural networks. However, the approaching physical limitations of electronic transistors have led to new innovative computing paradigms, with photonic computing gaining increasing attention. Photonic computing uses photons to perform mathematical calculations at the speed of light, offering the potential for faster and more energy-efficient processing.

Various photonic neural network architectures have been proposed in the past, including photonic spike processing, optical reservoir computing, and optical scatter materials. Two significant approaches in implementing large-scale interconnections of neurons between hidden layers are optical diffraction and optical interference. Despite the successes of on-chip photonic neural networks in AI tasks such as object classification and salient object detection, model performance heavily relies on the number of hidden layers (depth). Deeper neural networks have been associated with improved performance. However, the deployment of deeper networks on photonic integrated circuits requires larger chip areas. Hence, there is a need to optimize model performance within the constraints of limited chip area size.

In a new study published in the peer-reviewed Journal Optics Letters, Yun Zhao, Hang Chen, Min Lin, Haiou Zhang, Tao Yan, Ruqi Huang, Xing Lin, and Qionghai Dai from Tsinghua University, introduced a novel on-chip photonic neural network architecture based on the dynamic model, which is called the Optical Neural Ordinary Differential Equations (ON-ODEs). This architecture leverages the dynamic model of ordinary differential equations (ODEs) to parameterize the continuous dynamics of hidden layers with optical ODE solvers. The ON-ODEs aim to enhance model performance while reducing the chip area occupancy, making them a promising candidate for future photonic computing processors.

The research team proposed a novel on-chip photonic neural network architecture termed Optical Neural Ordinary Differential Equations (ON-ODEs). This architecture combines photonic neural networks (PNNs) with integrators and an optical feedback loop. The continuous-time dynamics of hidden layers are parameterized using ODEs, which enables sharing of parameters among different layers and reduces model size while maintaining higher memory efficiency. “The ON-ODE architecture is versatile and can be configured to implement optical residual neural networks (ResNets) and recurrent neural networks for various machine learning tasks,” according to corresponding author, Professor Xing Lin. The authors verified architecture with both Mach–Zehnder interferometer (MZI)-based optoelectronic nonlinear PNNs and diffractive photonic computing units (DPU)-based all-optical linear PNNs.

The researchers conducted elegant numerical experiments to evaluate the effectiveness of the ON-ODE architecture compared to conventional PNNs. They used the MNIST handwritten digit dataset for image classification tasks and tested trajectory prediction with high accuracy. The authors’ findings showed that the ON-ODE with one hidden layer achieves comparable accuracy to a two-layer optical ResNet, demonstrating its effectiveness in reducing chip area occupancy while maintaining performance. Additionally, the ON-ODE with DPU-based PNNs outperforms pure DPU-based PNNs in image classification tasks, highlighting its advantages in computing speed and energy efficiency.

In conclusion, the presented ON-ODE architecture by Professor Xing Lin and colleagues is a significant advancement in the field of photonic computing, offering enhanced model performance while reducing chip area occupancy. Moreover, the concept of parameterizing continuous dynamics with optical ODE solvers opens new possibilities for faster and more energy-efficient photonic neural networks. Furthermore, the experimental evaluations demonstrate the versatility and effectiveness of the ON-ODE in various machine learning tasks, making it a promising candidate for future on-chip implementations. Further research and development in this area hold great potential for revolutionizing AI computing with the power of photons.