Bridging Model Fidelity and Deep Learning: A Robust Reconstruction Framework for Lensless Imaging

Lensless imaging offers a way to capture images without bulky lenses which makes devices smaller, cheaper, and more versatile. This is a big deal for fields like virtual reality, augmented reality, and even the Internet of Things, where space and cost matter a lot. Instead of relying on traditional lenses to focus light, lensless imaging uses a coded mask to gather spatial information and then with the help of computational techniques, reconstructs high-quality images. The idea of skipping the lens sounds simple, but in reality, pulling off a clear and accurate image is not trivial. One of the biggest challenges in lensless imaging comes from how light spreads across the sensor. Each point of light from an object does not just hit a single spot; it gets scattered across the sensor in a way that follows a pattern known as the point spread function. Because of this, what the sensor captures is more like a jumbled mess of overlapping signals rather than a clean image. The job of reconstructing the original scene from this mess is a tricky inverse problem, meaning there is no straightforward way to simply “undo” the scattering. Scientists try to use a mathematical model—called a forward model—to predict how the light should behave, but  there were limitations of calibration errors and sensor noise. These mismatches between theory and reality cause distortions, blurriness, and frustrating artifacts in the final image. To tackle these issues, researchers first turned to deep learning, specifically convolutional neural networks (CNNs) to teach computers how to recognize patterns in raw sensor data and turn them into clearer images. These machine learning models worked fast and sometimes produced surprisingly good results. However, they had a major flaw—they did not always stick to the rules of physics. This meant that, while they might create a sharp image in one situation, they could completely fail when conditions changed. Recognizing this problem, scientists started combining physics-based models with deep learning to create hybrid approaches. One of the most promising ideas was “unrolling” traditional optimization algorithms into deep neural networks. This approach allowed the system to maintain both data consistency and efficiency, producing better and more reliable results.

Still, even these advancements had their limitations. Many models assumed that the theoretical imaging setup was perfect, ignoring the inevitable noise and errors that pop up in real-life conditions. On the other hand, deep learning-based approaches relied too much on training data, which did not always cover all possible scenarios. To this account, new research paper published in Optics Express and conducted by Associate Professor Hui Qian, graduate student Hong Ling, and Professor XiaoQiang Lu from the College of Physics and Information Engineering at Fuzhou University, developed the MN-FISTA-Net, a deep learning-based framework designed to tackle both model mismatch and noise at the same time. Unlike conventional methods that treat these issues separately, their approach filters out errors dynamically while ensuring that noise does not compromise image quality. They built their model using a Fast Iterative Shrinkage/Thresholding Algorithm (FISTA), a smart optimization technique that refines images over multiple steps. By integrating a CNN-based denoiser between the gradient descent and momentum modules, the system adapts on the fly, producing images that are cleaner, sharper, and more accurate than anything previous methods could achieve.

First the researchers built a lensless imaging prototype using a coded mask system which allowed them to gather raw data and put their model to the test. They tested it on a mix of publicly available ones—like FlatCam and DiffuserCam—as well as a custom dataset they built themselves. Each dataset had its own quirks, with different levels of noise and accuracy which make them the perfect challenge for assessing how robust and adaptable their approach really was. One of the biggest takeaways from their tests came from comparing MN-FISTA-Net with other well-known imaging methods, like FISTA-Net, Flatnet, MWDN-CPSF, and U-Net. Older techniques, like FISTA-Net, tended to be slow and struggled with noise which led to blurry and unreliable images. On the other hand, deep learning models like U-Net did a great job producing sharp images, but they often messed up the structure of objects because they lacked an understanding of how light behaves physically. The researchers found that MN-FISTA-Net outperformed them all, especially in situations where there were major differences between the expected model and real-world conditions.  The author’s approach of combining physics-based modeling with deep learning kept edges crisp, recovered fine textures, and improved overall image clarity. Moreover, the authors found that MN-FISTA-Net can handle different levels of noise, this is due to its built-in deep denoiser, which did not just try to fix errors after they appeared, but filtered them out before they could even affect the final image. Compared to other hybrid methods that relied on patchy compensation techniques, this filtering approach made a huge difference. The results spoke for themselves—images processed with MN-FISTA-Net had a higher Peak Signal-to-Noise Ratio. At the same time, they scored lower on the Learned Perceptual Image Patch Similarity scale, which meant they looked more natural and visually pleasing. Another major test focused on real-world objects, captured using their custom-built lensless imaging system which demonstrated its success in reconstructing images with sharper edges and more clearly defined shapes. According to the authors they credited this to their unique combination of mixed norm regularization and deep neural network priors which allowed the system to adjust to subtle differences in lighting and scene complexity. Speed was another big factor they looked into. Some high-end deep learning models, like MWDN-CPSF, could create high-quality images but at the cost of huge computational power. While MN-FISTA-Net did require more training than some simpler methods, it struck a solid balance between efficiency and accuracy. It was able to deliver high-quality reconstructions without the heavy processing burden that weighed down other models. In short, it was a practical and scalable solution that could be used in real-world imaging systems.

In conclusion, the research work of Professor XiaoQiang Lu and colleagues on lensless imaging has the potential to change how we think about capturing high-quality images in challenging conditions. With their innovative MN-FISTA-Net, the researchers have tackled a problem that has been frustrating scientists for years. One of the most exciting aspects of this work is what it could mean for compact imaging technologies. Lensless cameras do away with bulky lenses, making them perfect for devices where space is limited—think wearable tech, medical imaging tools, or remote sensing systems. The improvements made by MN-FISTA-Net could make these systems even sharper and more reliable, allowing for miniature cameras that deliver crisp, accurate images even in low-light or high-noise environments. This is especially promising for healthcare, where high-quality imaging is critical, but the equipment is often expensive or too large to be used in certain settings.  Another big impact of this research is in the world of AI-powered imaging. As machine vision keeps advancing, precise image reconstruction is becoming more important than ever, particularly for things like self-driving cars, augmented reality, and smart surveillance. The ability to restore fine details while reducing noise—without slowing down the process—could make MN-FISTA-Net a great addition to real-time imaging systems. This means faster, clearer, and more reliable image processing for a wide range of applications.