Intrinsically Bayesian robust Karhunen-Loève compression


In signal processing, different applications such as classification and filtering require the design of robust operators. Although the general design rule is the construction of a mathematical model followed by finding an operator that minimizes the cost function with respect to the desired design objectives, the challenge arises in cases involving uncertain models. The solution to such a problem was first conceived in 1977 when a linear filter was designed to maximize its robustness in the presence of uncertain covariance model. Consequently, the Bayesian framework involves finding an operator with the best performance by assuming a distribution over the uncertainty class: the intrinsically Bayesian robust (IBR) operator.

Generally, model uncertainty negatively affects the operator performance. The extent of the uncertainty can be quantified by using the mean objective cost of uncertainty (MOCU). The MOCU measures the average loss of performance by using the IBR operator instead of the actual optimal operator.

Karhunen-Loève (KL) compression has been utilized in many engineering applications to reduce complexity. This is due to its having a certain optimal property. However, its use is limited by the need to fully determine the second-order statistics and covariance matrix, which can be expensive. Therefore, researchers have been looking for an alternative method via manipulating the uncertainty class to obtain good compression.

Recently, researchers at Texas A&M University (Department of Electrical and Computer Engineering), Dr. Roozbeh Dehghannasiri (currently a post-doctoral research fellow in the Department of Biochemistry and Center for Cancer Systems Biology in Stanford School of Medicine, Stanford University), Professor Xiaoning Qian and distinguished Professor Edward Dougherty developed an intrinsically Bayesian robust Karhunen-Loève (IBR KL) compression when the unknown covariance matrix belongs to an uncertainty class of covariance matrices. The authors proposed to prove that by utilizing the IBR KL method, it is possible to minimize the expected MSE over the uncertainty class and also to solve the experimental design problem. Their research work is currently published in the research journal, Signal Processing.

The research team commenced their study by choosing the best covariance matrix to use for compression, and proved that IBR KL compression minimizes the MSE over the uncertainty class for m-term KL expansions. Furthermore, they solved the experimental design problem by determining the unknown covariance that maximally reduces the mean objective cost of uncertainty. The analytical solution of the optimal experimental design is solved through Wishart prior distributions. Furthermore, experimental simulations were carried out to verify the merits of IBR KL compression and applying experimental design repeatedly.

KL compression can be generally described as a random process having reduced complexity represented in a standard form. Thus, according to the authors, IBR KL is a robust generalization of the initial KL expansion based on the expected covariance matrix over an uncertainty class. Therefore, it overcomes the limitation of the KL compression that depends on knowing the true covariance matrix. Additionally, a very important advantage is that the experimental design reduces uncertainty relative to the specific area of application. Considering the accuracy and effectiveness of their study, it is expected to advance various engineering applications like signal processing through much improved operator design.

According to Prof. Dougherty, the basic theory can be extended to many application domains, such as therapeutic intevention in gene regulatory networks and materials design. Several months ago, Prof. Dougherty published a book discussing a wide range of applications: Optimal Signal Processing Under Uncertainty (SPIE Press, 2018).

Intrinsically Bayesian robust Karhunen-Loève compression. Advances in Engineering


About the author

Roozbeh Dehghannasiri received the B.S. degree from the University of Tehran, Tehran, Iran, in 2010, the M.A.Sc. degree from the McMaster University, Hamilton, Canada, in 2012, and the Ph.D. degree from Texas A&M University, College Station, TX, USA, in 2016, all in electrical engineering. Upon his graduation, he became a Post-Doctoral Research Associate in the Department of Electrical and Computer Engineering, Texas A&M University, where he worked on uncertainty analysis in gene regulatory networks and designing Bayesian robust filters for uncertain random process models. Since 2017, he is a Post-Doctoral Research Fellow in the Department of Biochemistry and Center for Cancer Systems Biology in Stanford School of Medicine, Stanford University.

His current research interests include statistical detection of gene fusions using RNA-Seq and designing integrative models for cancer genomics data analysis. Dr. Dehghannasiri received the Best Paper Award at the 12th Annual Mid-South Computational Biology and Bioinformatics Conference in 2015, the Best paper award at the International Conference on Intelligent Biology and Medicine in 2018, and the McMaster Outstanding Thesis Research Award in 2012.

About the author

Xiaoning Qian received the Ph.D. degree in electrical engineering from Yale University, New Haven, CT, USA. He is currently an Associate Professor with the Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA. He is also with the TEES-AgriLife Center for Bioinformatics and Genomic Systems Engineering (CBGSE) and the Center for Translational Environmental Health Research (CTEHR) at Texas A&M University. His research interests include machine learning and Bayesian computation with their applications in computational network biology, genomic signal processing, and biomedical signal and image analysis. He was a recipient of the National Science Foundation CAREER Award, the Texas A&M Engineering Experiment Station Faculty Fellow, and the Montague-Center for Teaching Excellence Scholar at Texas A&M University.

His recent work on computational network biology has received the Best Paper Award at the 11th Asia Pacific Bioinformatics Conference in 2013 and the Best Paper Award in the International Conference on Intelligent Biology and Medicine in 2016.

About the author

Edward R. Dougherty is a Distinguished Professor in the Department of Electrical and Computer Engineering at Texas A&M University in College Station, TX, where he holds the Robert M. Kennedy ‘26 Chair in Electrical Engineering and is Scientific Director of the Center for Bioinformatics and Genomic Systems Engineering. He holds a Ph.D. in mathematics from Rutgers University and an M.S. in Computer Science from Stevens Institute of Technology, and has been awarded the Doctor Honoris Causa by the Tampere University of Technology.

He is a fellow of both IEEE and SPIE, has received the SPIE President’s Award, and served as the editor of the SPIE/IS&T Journal of Electronic Imaging. At Texas A&M University he has received the Association of Former Students Distinguished Achievement Award in Research, been named Fellow of the Texas Engineering Experiment Station and Halliburton Professor of the Dwight Look College of Engineering. Prof. Dougherty is author of 19 books and author of more than 350 journal papers.


Dehghannasiri, R., Qian, X., & Dougherty, E. (2018). Intrinsically Bayesian robust Karhunen-Loève compression. . Signal Processing, 144, 311-322. .

Go To Signal Processing

Check Also

Featured Video Play Icon

Terahertz generation by spatiotemporally patternable optical pumping beam