Generally, modeling and simulation is a key enabler for systems engineering activities as the system representation in a computer readable model enables engineers to reproduce the system (or Systems of System) behavior. Over the years, surrogate-based global optimization has played an increasingly important role in many areas of engineering and science. Typically, by constructing surrogate models to approximate or take place of the time-consuming computer simulations, the design efficiency can be impressively improved. Many representative surrogate models have been developed, among which kriging modeling stands out. Kriging, also known as Gaussian process, is widely used in computationally expensive optimization problems to improve the design efficiency. Although the basic theory of kriging has almost been developed for seven decades, it still suffers from some drawbacks for high-dimensional problems. For instance, for one to find the global optimum, a large amount of computational demand is needed. This shortfall is colloquially termed “curse-of-dimensionality” and results in exponential increase in time as the dimension of the problem grows.
In fact, when it comes to the cases where by the kriging model needs to be frequently constructed, such as: sequential sampling for kriging modeling or global optimization based on kriging model, the increased modeling time ought to be taken into consideration. Therefore, to address this challenge, researchers from the Northwestern Polytechnical University, Xi’an in China: Mr. Liang Zhao, Professor Peng Wang, Professor Baowei Song, Dr. Xinjing Wang and Dr. Huachao Dong, proposed a novel kriging modeling method that could combine kriging with maximal information coefficient (MIC). Their goal was to develop a novel kriging modeling method which could build a kriging model with a little amount of computational effort for high- dimensional problems. Their work is currently published in the research journal, Structural and Multidisciplinary Optimization.
In their work, the team took into consideration the features of the optimized hyper-parameters and utilized MIC to estimate the relative magnitude of hyper-parameters. The researchers then incorporated the pre-acquired knowledge of hyperparameters into the maximum likelihood estimation problem to reduce the dimensionality. That way, the high dimensional optimization could be transformed into a one-dimensional optimization, which could in turn significantly improve the modeling efficiency. Overall, five representative numerical examples from 20-D to 80-D and an industrial example with 35 variables were used to show the effectiveness of the proposed method.
The authors reported that compared to conventional kriging, the modeling time of the proposed method could be ignored, while the loss of accuracy was acceptable. Better still, for the problems with more than 40 variables, the proposed method could even obtain a more accurate kriging model with given computational resources. Additionally, results form comparison with KPLS (kriging combined with the partial least squares method) revealed that the proposed method was more competitive hence more efficient kriging modeling method for high-dimensional problems.
In summary, the study presented a novel kriging modeling method (KMIC) that is better suited for high-dimensional design problems. Generally, the proposed method combined kriging with maximal information coefficient (MIC) and thus termed KMIC. MIC was used to estimate the relative magnitude of the optimized hyper-parameters because both the optimized hyperparameters and MIC can be used for global sensitivity analysis. In a statement to Advances in Engineering, Professor Peng Wang emphasized that with their approach, only one auxiliary parameter was needed to optimize when estimating hyper-parameters. As a consequence, the modeling efficiency was dramatically improved.
Liang Zhao, Peng Wang, Baowei Song, Xinjing Wang, Huachao Dong. An efficient kriging modeling method for high-dimensional design problems based on maximal information coefficient. Structural and Multidisciplinary Optimization (2020) volume 61: page 39–57.