Fossil fuels  continue to account for eighty-five per cent of the world’s energy consumption. The need for renewable energy resources is unequivocal. As a result, we have recently witnessed an increase in renewable energy penetration, amounting to ~27% of global electricity generation in 2019 . However, the ability of the existing electric grid to support renewable energy penetration is limited as the infrastructure was designed to support unidirectional power-flows emanating from a few generating sources to a large number of consumers. Consequently, power-flows from distributed renewable energy sources pose grid stability challenges. In addition, growing concerns regarding efficiency, reliability, and security necessitate the transformation of the present grid into a “smart grid”.
The smart grid paradigm is envisioned to integrate several forward-looking technologies while maintaining full backward compatibility with the existing grid without compromising stability. Information from various sensors are integrated via communication networks to enable intelligent real-time power-flow decisions. Several studies reported in the scientific literature have examined smart grid decision problems, especially in the context of microgrids and nanogrids. However, most of these works either do not account for important nonlinearities and/or do not accommodate uncertainties in supply and demand.
To this end, a joint effort by researchers from Carnegie Mellon University (CMU SV) and Cisco Systems Inc. investigated the design of optimal energy management policies in the presence of supply and demand uncertainties. The research conducted by Dr. Chaitanya Poolla* under the guidance of Dr. Abraham K. Ishihara# from CMU SV and Dr. Rodolfo Milito↟ from Cisco Systems Inc. resulted in a weather forecast-integrated Markov Decision Process-based (MDP) framework for near-optimal policy design in the presence of uncertainties. Their work is published in the scientific journal, Applied Energy .
To address the energy management problem, a modeling framework consisting of battery dynamics, a stochastic demand model, a stochastic solar generation model, and an electricity pricing scheme was considered as shown in Figure 1. Within this framework, the researchers proposed MDP-based near-optimal policies and contrasted their performance to that of rule-based policies. Based on simulations in residential and commercial environments, the authors found the MDP approach significantly reduced operating costs in comparison to the heuristic alternatives.
This work has been identified by the Advances in Engineering (AIE) selection committee as a significant contribution to research excellence in the field of energy. In a statement to AIE, Dr. Chaitanya Poolla highlighted that the proposed framework facilities the design and evaluation of energy management policies with configurable demand-supply-storage parameters in the presence of weather-induced uncertainties.
* Dr. Chaitanya Poolla is with Intel Corporation. This work was performed when he was a graduate student at Carnegie Mellon University.
# Dr. Abraham K. Ishihara is with KBR. This work was performed when he was with Carnegie Mellon University.
↟ Dr. Rodolfo Milito is with Starflow Networks Inc. This work was performed when he was with Cisco Systems Inc.
 Dudley, Bob. BP statistical review of world energy. BP Statistical Review, London, UK, June (2019): 2019.
 IEA (2019), Tracking Power, IEA, Paris https://www.iea.org/reports/tracking-power-2019