Fossil fuels [1] continue to account for eighty-five per cent of the world’s energy consumption. The need for renewable energy resources is unequivocal. As a result, we have recently witnessed an increase in renewable energy penetration, amounting to ~27% of global electricity generation in 2019 [2]. However, the ability of the existing electric grid to support renewable energy penetration is limited as the infrastructure was designed to support unidirectional power-flows emanating from a few generating sources to a large number of consumers. Consequently, power-flows from distributed renewable energy sources pose grid stability challenges. In addition, growing concerns regarding efficiency, reliability, and security necessitate the transformation of the present grid into a “smart grid”.

The smart grid paradigm is envisioned to integrate several forward-looking technologies while maintaining full backward compatibility with the existing grid without compromising stability. Information from various sensors are integrated via communication networks to enable intelligent real-time power-flow decisions. Several studies reported in the scientific literature have examined smart grid decision problems, especially in the context of microgrids and nanogrids. However, most of these works either do not account for important nonlinearities and/or do not accommodate uncertainties in supply and demand.

To this end, a joint effort by researchers from Carnegie Mellon University (CMU SV) and Cisco Systems Inc. investigated the design of optimal energy management policies in the presence of supply and demand uncertainties. The research conducted by Dr. Chaitanya Poolla* under the guidance of Dr. Abraham K. Ishihara# from CMU SV and Dr. Rodolfo Milito↟ from Cisco Systems Inc. resulted in a weather forecast-integrated Markov Decision Process-based (MDP) framework for near-optimal policy design in the presence of uncertainties. Their work is published in the scientific journal, Applied Energy [3].

To address the energy management problem, a modeling framework consisting of battery dynamics, a stochastic demand model, a stochastic solar generation model, and an electricity pricing scheme was considered as shown in Figure 1. Within this framework, the researchers proposed MDP-based near-optimal policies and contrasted their performance to that of rule-based policies. Based on simulations in residential and commercial environments, the authors found the MDP approach significantly reduced operating costs in comparison to the heuristic alternatives.

This work has been identified by the Advances in Engineering (AIE) selection committee as a significant contribution to research excellence in the field of energy. In a statement to AIE, Dr. Chaitanya Poolla highlighted that the proposed framework facilities the design and evaluation of energy management policies with configurable demand-supply-storage parameters in the presence of weather-induced uncertainties.

* Dr. Chaitanya Poolla is with Intel Corporation. This work was performed when he was a graduate student at Carnegie Mellon University.
# Dr. Abraham K. Ishihara is with KBR. This work was performed when he was with Carnegie Mellon University.
Dr. Rodolfo Milito is with Starflow Networks Inc. This work was performed when he was with Cisco Systems Inc.

Figure 1: A schematic representation of the framework depicting the weather forecast system, solar generation model, demand model, pricing scheme, battery model, and the optimization framework

About the author

Chaitanya Poolla works as a software engineer/data scientist at Intel Corporation on problems related to computer performance and power. He received a B.Tech (Honors) degree in Aerospace Engineering from the Indian Institute of Technology – Kharagpur in 2011, and MS and PhD degrees in Electrical and Computer Engineering from Carnegie Mellon University in December 2016.

He has contributed to more than ten scientific publications in internationally recognized journals and conferences, and is a recipient of a NASA honor award, BOEING fellowships, IIT silver medal for best graduating student in Aerospace engineering, and gold & silver medals in Indian national mathematics Olympiads.

He is interested in topics such as machine learning, optimization, design of experiments, statistical inference, decision sciences, dynamics, optimal & adaptive control. His work is primarily concerned with addressing challenges faced by the computing, energy, and transportation industries. In addition to his academic and professional activity, he is interested in philosophy (ontology and epistemology), science, and metaphysics.

About the author

Rodolfo Milito got his PhD in EE (Stochastic Control Systems) from the University of Illinois at Urbana-Champaign (UIUC), where he acquired a solid background in stochastic systems, statistical analysis, and learning systems. Following graduation, he joined Bell Laboratories in 1985, and AT&T Labs after the 1996 Lucent spin-off. He worked on mathematical model and performance characterization of networks, and algorithms to improve their performance (load distribution, routing, resource sharing, overload control). In 1999 he moved to the Silicon Valley to join the startup XStream Logic, and later cofounded Consentry Networks (focused on malware detection).

Rodolfo joined Cisco in February 2008 and left it in June 2017 to join Starflow Networks/Clevernet. At Cisco he contributed to IoT (Smart Cities, Precision Agriculture, Healthcare, Connected Conservation), did pioneering work on Big Data & Analytics, and “Fog Computing”, the distributed compute, storage, and networking architecture that extends from the edge to the Cloud.

Rodolfo has published extensively https://scholar.google.com/citations?user=WnHiD3gAAAJ&hl=en, and holds 12 US patents. The paper he presented at SIGCOMM in 2012 (“Fog Computing and its Role in IoT” has been cited over 3500 times.


[1] Dudley, Bob. BP statistical review of world energy. BP Statistical Review, London, UK, June (2019): 2019.

[2] IEA (2019), Tracking Power, IEA, Paris https://www.iea.org/reports/tracking-power-2019

Go To IEA, Paris

[3] Chaitanya Poolla, Abraham K. Ishihara, Rodolfo Milito. Designing near-optimal policies for energy management in a stochastic environment. Applied Energy, volume 242 (2019) page 1725–1737.

Go To Applied Energy

Check Also

Bacteria hide in their sturdy house to survive the harsh environment - Advances in Engineering

Bacteria hide in their sturdy house to survive the harsh environment