Intrinsically Bayesian robust Karhunen-Loève compression

In signal processing, different applications such as classification and filtering require the design of robust operators. Although the general design rule is the construction of a mathematical model followed by finding an operator that minimizes the cost function with respect to the desired design objectives, the challenge arises in cases involving uncertain models. The solution to such a problem was first conceived in 1977 when a linear filter was designed to maximize its robustness in the presence of uncertain covariance model. Consequently, the Bayesian framework involves finding an operator with the best performance by assuming a distribution over the uncertainty class: the intrinsically Bayesian robust (IBR) operator.

Generally, model uncertainty negatively affects the operator performance. The extent of the uncertainty can be quantified by using the mean objective cost of uncertainty (MOCU). The MOCU measures the average loss of performance by using the IBR operator instead of the actual optimal operator.

Karhunen-Loève (KL) compression has been utilized in many engineering applications to reduce complexity. This is due to its having a certain optimal property. However, its use is limited by the need to fully determine the second-order statistics and covariance matrix, which can be expensive. Therefore, researchers have been looking for an alternative method via manipulating the uncertainty class to obtain good compression.

Recently, researchers at Texas A&M University (Department of Electrical and Computer Engineering), Dr. Roozbeh Dehghannasiri (currently a post-doctoral research fellow in the Department of Biochemistry and Center for Cancer Systems Biology in Stanford School of Medicine, Stanford University), Professor Xiaoning Qian and distinguished Professor Edward Dougherty developed an intrinsically Bayesian robust Karhunen-Loève (IBR KL) compression when the unknown covariance matrix belongs to an uncertainty class of covariance matrices. The authors proposed to prove that by utilizing the IBR KL method, it is possible to minimize the expected MSE over the uncertainty class and also to solve the experimental design problem. Their research work is currently published in the research journal, Signal Processing.

The research team commenced their study by choosing the best covariance matrix to use for compression, and proved that IBR KL compression minimizes the MSE over the uncertainty class for m-term KL expansions. Furthermore, they solved the experimental design problem by determining the unknown covariance that maximally reduces the mean objective cost of uncertainty. The analytical solution of the optimal experimental design is solved through Wishart prior distributions. Furthermore, experimental simulations were carried out to verify the merits of IBR KL compression and applying experimental design repeatedly.

KL compression can be generally described as a random process having reduced complexity represented in a standard form. Thus, according to the authors, IBR KL is a robust generalization of the initial KL expansion based on the expected covariance matrix over an uncertainty class. Therefore, it overcomes the limitation of the KL compression that depends on knowing the true covariance matrix. Additionally, a very important advantage is that the experimental design reduces uncertainty relative to the specific area of application. Considering the accuracy and effectiveness of their study, it is expected to advance various engineering applications like signal processing through much improved operator design.

According to Prof. Dougherty, the basic theory can be extended to many application domains, such as therapeutic intevention in gene regulatory networks and materials design. Several months ago, Prof. Dougherty published a book discussing a wide range of applications: Optimal Signal Processing Under Uncertainty (SPIE Press, 2018).