DeepSEM-Net: A Hybrid Model for Accurate, Efficient Defect Detection in Semiconductor Manufacturing

In today’s world of tech innovation, semiconductors are absolutely essential. From our phones and laptops to sophisticated AI systems and self-driving vehicles, these tiny circuits power almost everything. But as integrated circuits get smaller and more complex, manufacturing them without a hitch has become increasingly tricky. One of the major headaches for semiconductor makers is figuring out how to detect and categorize microscopic defects on silicon wafers—the foundational pieces for these circuits. If even the smallest flaw slips through during manufacturing, it could seriously undermine the performance and reliability of the final device. And when you think about how many gadgets rely on these components, it becomes clear why it’s crucial to improve defect detection methods. To stay on top of quality control, semiconductor manufacturers use scanning electron microscopes, or SEMs. These powerful microscopes generate detailed images of the wafer surfaces, giving engineers a close look at any imperfections. But even with SEMs, analyzing these images accurately is no walk in the park. They contain all kinds of textures and subtle defect types that traditional inspection methods struggle to categorize. Typically, fabs rely on rule-based systems or manual inspections, which are not only slow but often lead to human errors. The sheer volume of data produced in semiconductor plants also means that these older methods just can’t keep up. Machine learning seems like an ideal answer, but even that comes with challenges—like needing massive amounts of correctly labeled data to train the algorithms, which isn’t always available. That’s where Professor Yining Chen and co-authors (Dr. Yibo Qiao, Dr.  Zhouzhouzhou Mei, Dr.  Yuening Luo) at Zhejiang University stepped in. Seeing these issues, they aimed to build something more advanced. Their work, published in Computers & Industrial Engineering, developed a deep learning model they’re calling DeepSEM-Net. What makes DeepSEM-Net special is that it combines the best of two methods: Convolutional Neural Networks (CNNs) and Transformers. CNNs excel at spotting small, local features in images, while Transformers bring a broader context, capturing the bigger picture. By blending these two approaches, DeepSEM-Net can detect and classify wafer defects in SEM images with impressive speed and accuracy. It’s a promising step forward, offering an automated solution that fits well with the demands of semiconductor manufacturing today.

To thoroughly evaluate DeepSEM-Net’s capabilities, the research team set up a series of experiments using SEM images of semiconductor wafers, focusing on common defects that appear during manufacturing. Their goal was to test how well the model could detect, classify, and precisely outline these flaws compared to standard methods and other deep learning models. Given that semiconductor manufacturing often deals with limited labeled data, they also wanted to see how DeepSEM-Net performed under these real-world conditions, where labeled examples are not always easy to come by. For these experiments, the authors pulled together a large set of SEM images, carefully selected to represent a wide range of defect types and sizes, to make sure they could get a realistic sense of the model’s reliability. They split the images into training and testing groups, setting up the model to learn from one set and then apply that learning to new, unseen images. This allowed them to check if DeepSEM-Net could generalize and accurately identify defects it hadn’t seen before. They also measured its performance against other commonly used models, such as traditional CNNs and Transformer-only models, to see if their hybrid approach would show clear advantages in recognizing complex defect patterns.

One of the standout results was DeepSEM-Net’s higher accuracy and stability when compared to conventional CNN models. Thanks to its combination of CNN layers, which capture minute, localized details, and Transformer modules, which bring in broader contextual understanding, DeepSEM-Net could detect even small, intricate defects with impressive precision. When the team compared accuracy across models, they found that DeepSEM-Net consistently came out on top, especially when it came to identifying defects with complex textures and patterns. This confirmed their hypothesis that a model blending CNN and Transformer elements would excel at the detailed analysis needed in SEM images. A major part of the testing involved seeing how well DeepSEM-Net would hold up with limited labeled data—a common scenario in semiconductor inspection because labeling SEM images by hand can be time-consuming and labor-intensive. To simulate this, the researchers restricted the amount of labeled data available for training, observing how the model adapted to fewer examples. Even with a smaller set of labeled images, DeepSEM-Net maintained high accuracy, likely due to its efficient architecture that allows for better learning and generalization from limited data. Traditional CNN models, on the other hand, showed a significant drop in accuracy under the same conditions, which highlighted DeepSEM-Net’s robustness for real-world applications where labeled data is often scarce.

The team also looked at DeepSEM-Net’s segmentation abilities—an important feature for identifying defect boundaries accurately, which helps in assessing their impact on the wafer. Through pixel-level segmentation, DeepSEM-Net could outline defects with a clarity that previous models struggled to reach. This level of detail is crucial for assessing the seriousness and type of defect involved. The segmentation results were promising; DeepSEM-Net accurately mapped out complex shapes, underscoring its ability to not just detect and classify defects but to provide precise spatial information, which is invaluable for pinpointing issues in the production process. Recognizing that speed is key in industrial settings, the researchers assessed DeepSEM-Net’s computational efficiency as well. Despite its complex architecture, the model processed images quickly, performing at speeds comparable to simpler CNNs. This is due to its optimized combination of CNN and Transformer components, which allows it to handle high-resolution SEM images swiftly. The fact that DeepSEM-Net managed to balance both speed and accuracy means it could be realistically integrated into real-time inspection systems, keeping up with the high-throughput demands of semiconductor production without slowing things down.

The value of Professor Yining Chen and colleagues study lies in its potential to reshape quality control in semiconductor manufacturing, a field where pinpoint accuracy in detecting even the tiniest defects is crucial to ensuring the reliability of high-tech components. By introducing DeepSEM-Net, a sophisticated model that blends the strengths of CNN and Transformer architectures, this research offers a new method that doesn’t just surpass conventional models in spotting and mapping out complex defects in SEM images—it also performs well even when labeled data is limited, a frequent hurdle in the industry. Essentially, this innovation allows manufacturers to maintain high accuracy in defect detection without needing time-intensive, extensive labeling efforts, which has practical benefits for both productivity and cost in real-world applications. One of DeepSEM-Net’s standout qualities is its adaptability to the high-speed, high-volume demands of inspection lines. Its capability to quickly and accurately process high-resolution images while delivering precise defect segmentation is especially valuable in semiconductor production, where even tiny flaws can lead to significant issues, like yield losses or device malfunctions. By making it possible to spot defects faster and with more accuracy, this model supports manufacturers in their efforts to improve output and reduce waste, which ultimately contributes to more sustainable manufacturing practices. Another important takeaway is the potential of this model to raise the bar for automated quality control in fields where microscopic inspection is essential. Thanks to its hybrid architecture, DeepSEM-Net has applications that could go beyond semiconductors into areas like biomedical imaging or materials science, where identifying fine details and subtle anomalies is critical. The success of this model points to a larger trend towards hybrid machine learning architectures, showing how combining CNN’s ability to capture fine details with Transformer’s broader contextual awareness can create versatile tools for increasingly complex industrial demands.