Three-layer informatics solution for demand response [612739]

Three-layer informatics solution for demand response
management in the smart grid context
Simona-Vasilica Oprea1, Adela Bâra1 and Osman Bulent Tor2
1The Bucharest University of Economic Studies, Romania
2EPRA Engineering Procurement Research Analysis, Ankara, Turkey
E-mail: [anonimizat]

Abstract. In consistency with the European Union (EU) vision of creating an electric power
system that integrates renewables and enables flexible consumption and generation
technologies by means of smart grid (SG) implementation, the main aim of the paper is to
develop different level of aggregation mechanisms design that facilitates demand response
(DR) and energy systems to utilize up to 100% renewable generation.
In this paper, we will present a three-layer informatics solution for demand response
management in the smart grid context. The data model will be developed as a scalable and
customized framework using open solutions based on cloud computing.

1. Introduction
Main challenge is to increase flexibility of the grid, complex interactions among large and local or
regional energy stakeholders with massive volumes of heterogeneous data (big data challenge).
Advancements in SG technologies make possible to apply various strategies for optimization of DR.
Its aggregation would essentially accumulate potential consumption schedules and constraints given
by small/medium-size consumers for the participation in wholesale electricity markets. However,
there are still barriers for widespread adoption of DR aggregation mechanisms at the EU level.
According to the Final 10-year ETIP SNET R&I roadmap 2017-26 , these include: differences
between market structures and legislation; limited awareness and involvement of electricity
consumers/prosumers; less motivating incentives for participants; ICT infrastructure at an affordable
and reasonable cost compared with benefits; requirements of big data analytics for DR; interactions
between aggregators and Distribution System Operators (DSOs); lack of business models for market
trading [1].
Implementations of DR aggregator solutions vary over EU countries due to different market
structures and development. Advancements in SG technologies enable various options/strategies for
the DR optimization in electricity markets. However, there are still barriers for the widespread
adoption of DR aggregation mechanisms at the EU level, as emphasized in Final 10-year ETIP SNET
R&I Roadmap (2017-26) . The community aggregator (CA) concept has been implemented in Nordic
countries and can essentially be implemented in other countries (Turkey, Romania), where market
development (legislation, rules & policy) is behind compared to Nordic countries. Another concern
is about reliability of the ICT infrastructure coupling the consumers, market players and DSOs at an
affordable/reasonable cost compared with benefits. In response, our approach demonstrates that DR
can be categorized into different levels, from simple CA solution, which can be implemented in any

country in EU, to more complex wholesale electricity market aggregation concept for developed
markets. Our approach uses the hands-on experience in the CA concept, while recommending market
based DR solutions to expand in other countries. The idea of multi-layer aggregation (MLA) solutions
integrates a wider range of requirements to a scalable prototype that leads to better scalability and
transferability among the EU countries. Thereby, it has a better chance to disseminate the innovative
technologies based findings to a wider audience and markets across EU countries.
Energy Union Package 2015 emphasizes the requirement for moving away from an economy
driven by fossil fuels, based on a centralized, supply-side approach, relying on old technologies and
outdated business models [2]. This necessitates empowering consumers by providing information,
choice and flexibility to manage demand as well as supply. The European Commission is preparing
an ambitious legislative proposal to redesign the internal electricity market (IEM) to link wholesale
and retail. The electricity market will include many new producers, including renewables (RES), as
well as enable full participation of consumers via DR.
The authors of [3] approach the demand response management by proposing an optimal
community schedule of hourly electricity consumption considering advanced electricity tariffs such
as real-time tariff that flattens the daily consumption curve. The proposed algorithm minimizes the
electricity payment by considering the characteristics of electrical appliances and consumption
preferences of the residential consumers. It is based on several steps, taking into account
adjustable/programmable appliances, local generation sources, storage and common consumption at
the community level.
In [4], the authors propose a heuristic-based evolutionary approach based on shifting techniques
of several types of appliances at the communities level that treats different consumption sectors such
as: householders, commercial and industrial consumption.
Decision-making models for demand response aggregators that act on the wholesale electricity
markets are developed in [5]. The aggregators provide different types of contracts that aim to reduce
and optimize the electricity consumption. The hourly schedule of the appliances depends on the
electricity tariff rate and other provisions of the contracts.
In [6], the authors propose a demand response model bidding framework based on mixed-integer
linear programming at the Independent System Operator level that considers consumption preferences
and characteristics of the appliances. Demand response management includes measures such as load
shifting, curtailments, local generation and storage devices usage.
Comparing with these approaches related to demand response management, the informatics
solution proposed in this paper will be developed as an informatics prototype that aims to demonstrate
the implementation of DR aggregator solutions across EU countries which have different electricity
market structures, regulations and SG implementation stage. The prototype will manage massive data
collected from smart meters (SM) and appliances (IoT) for advanced analytics and will be developed
on a cloud computing (CC) platform which will enable ubiquitous access to distributed configurable
computing resources.
2. Community aggregator multi-layer solution
Due to various stages of development of SG implementation in different countries or regions, our
proposal addresses DR aggregating informatics solution in multi-layers beginning from the simplest
community aggregator (CA Layer 1) solution. Motivation of the CA will be optimal load scheduling
to minimize the community-aggregated electricity payment and consumption peak shaving
considering the convenience of individual residents and hourly community load characteristics.
Control of demand (i.e., active DR) including electric vehicle (EV) charging stations, distributed
renewable energy sources (RES) generation and storage devices (SD) will be addressed in Layer 2.
Finally, in Layer 3, aggregation of DR contracts will be considered in the price-based self-scheduling
optimization problem to determine optimal DR schedules for participants in day-ahead energy
markets as in Figure 1.

Figure 1. Multi-layer aggregation short description

Layer 1 can be the initial DR solution given its applicability to any community even without SG
implementation. Layer 2 may include several sub-layers depending on SG implementation level.
However, Layer 3 requires a DR bidding framework in day-ahead electricity markets, which
integrates consumers’ DR preferences and characteristics in the DSO’s market clearing process.
ENTSO/E and EDSO outline the main challenges connected with the DR from the grid perspective.
ENTSO/E R&D & Innovation Roadmap 2017–2026 highlights the need to activate the demand as a
new source/tool for system operation and to integrate it in the planning and operation stages and
market design; however it does not accentuates the tasks of activation of the source, leaving to DSOs,
aggregators or other players the possibility to do it [7]. In any case, integration of DR has been
identified as one of the outstanding issues to be dealt jointly with DSOs. Layer 3 addresses this issue.
The multi-layer aggregation approach covers different structures of markets and regulations as in
Figures 2, 3 and 4. It is more applicable since in Europe, the development of SG implementation can
significantly vary. For less developed regions, we propose a simple solution that monitors the
consumption and provide recommendations (Layer 1), while for more developed markets where
aggregators are involved in market clearing, we propose optimization and control solutions for DR
based on interaction between aggregators and DSOs (Layers 2 and 3) utilizing the block-chain based
smart contracts.

Figure 2. Layer 1 CA detailed concept

Figure 3. Layer 2 CA detailed concept

Figure 4. Layer 3 CA detailed concept
Our approach will develop a scalable cloud based platform to provide access to
consumers/prosumers and aggregators/DSOs to a complex data framework and models allowing big
data and real-time analytics including forecasting, optimization, profiling, and business models. We
will also approach the big data and CC challenges related to SG in order to increase consumer
awareness and improve the DR mechanisms. We will investigate and verify the high-reliability and
low-latency ICT technologies to support real-time monitor and sensing. Block-chain technologies
will be investigated to provide smart and flexible contracts. These aspects are among the most
prevailing barriers for the widespread adoption of DR aggregation mechanisms at the EU level, as
underlined by Final 10-year ETIP SNET R&I Roadmap and EC Energy Union Package 2015 .

3. Data framework
The data framework contains several tiers that can be customized based on each layer. These tiers
will allow complete customization, using different technologies for interconnecting sensors through
smart meters (SM-Tier); fast processing and real-time analyses through big data (BG-Tier); high-
reliable, low-latency, secure and performant data transmission and management with relational
databases (RB-Tier); historical and advanced analytics through data warehouse (DW-Tier) as in
Figure 5.

Figure 5. Tiers for each layer

The data framework will be developed on multi-tier architecture for high scalability, using open
standards for wider integration and adoption. For SM-Tier, we’ll consider OpenIoT, FIWARE & IoT-
A ARM platforms currently adopted and standardized in EU. To achieve real-time information
sensing and collection, we’ll develop and demonstrate ultra-reliability low-latency communications
based on short-packet. For BG Tier, we’ll consider optimization methods, such as: hash partitioning,
parallel execution, dimensionality reduction, Map Reduce. For load profiles (LP), we will use
artificial neural networks (ANN) algorithms (Self Organizing Maps, k-means, Fuzzy C-Means) for
clustering data across multiple nodes, with different type of SM, preserving consumers’ location,
weather, day type and dynamically adding particular characteristics. For forecasting, ANN algorithms
will be developed/run on both historical and online data for short term load forecast (LF).
Optimization algorithms will be developed for each layer. Layer 1 provides optimal load
scheduling of the community to minimize the electricity payment considering the convenience of
individual residents and hourly community load characteristics. Inputs include: LP, LF, cost of
electricity consumption, and preferences of the consumers in the community as constraints included
on the hourly utility load (defined as CA load minus the local generation). Lagrangian relaxation (LR)
will be applied to decouple the utility constraint and provide tractable sub-problems formulated as
mixed-integer programming (MIP).
Optimization in Layer 2 includes various contract offers for the customers for load curtailment
and shifting, local generation, managing electrical vehicles (EV) charging stations and storage
systems as possible strategies for hourly load reductions. The aggregation of DR contracts is
considered in the proposed price-based self-scheduling optimization model to determine optimal DR
schedules for participants in day-ahead energy markets. We’ll use block-chain to develop smart and
flexible contracts.
RB-Tier
DW-Tier
BG-Tier
STORAGE
MODELS
ANALYTYCS
INTEGRATION
KPIs
Real time analytics
Optimization
Forecast
Data Mining
SM-Tier: smart appliances,
generation, EVs

In Layer 3, a hierarchical DR bidding framework in the day-ahead energy markets, which
integrates consumers’ preferences and characteristics in the DSO’s market clearing process, will be
developed. Aggregators can submit offers to the DSO which will optimize final decisions on
aggregators’ DR contributions to wholesale market. The DSO applies MIP to the solution of the
proposed DR model in the day-ahead market clearing problem.
As an innovative concept, the prototype will be developed as a scalable-customized cloud
computing web-services using open standards, models and connectivity as in Figure 6.

Figure 6. Tiers for each layer

The prototype will be developed modular and will be configured based on the requirements of
stakeholders. Each proposed layer of the solution will have its innovative and original elements. The
prototype’s models will be developed on a data framework with multiple tiers (SM, BG, RB, DW
Tiers) providing reliable and consistent data for the models and algorithms including LP, LF and
electricity price from the utility, feed-in tariffs for RES, etc.
For integrating SM data, we’ll develop agents to connect sensors with different communication
protocols (ZigBee, LORA, WiFi, Wi-SUN) in order to load custom data in the BG-Tier. One of our
key innovations in is to support ultra-reliable low-latency communications based on short-packet for
supporting real-time SM data transmission and corresponding actuation, which is very important to
multi-layer systems. The platform can be extended to connect sensors from other utilities (gas,
heating) and enabling Advanced Metering Infrastructure (AMI).
The SM and BG Tiers will be used for real time analyses and for developing the algorithms for
controlling, monitoring & online forecasting and optimization. The SM-Tier will be implemented on
an edge computing architecture to perform data processing at the edge of the network, thus reducing
the communications bandwidth between SM and BG-Tier. Relational databases and big data are
complementary technologies that process data at an impressive rate of more than 10TB (terabytes)
per hour. Through our integration framework, we'll first process big data in the BG-Tier on a
distributed, scalable file system, such as HDFS (Hadoop File System) and develop query patterns
with processing engines (Hive, Drill, Impala or Presto). For data processing, we’ll develop algorithms
that are executed in parallel and extract relevant data based on a pattern matching criteria. These
patterns will be defined based on the requirements provided by the aggregator/DSO concerning their
analytical perspective. At the BG-Tier, we’ll develop forecasting algorithms based on ANN (feed-
forward, deep-learning). They will be developed online, receiving immediate feedback and improving
subsequent predictions in contrast to statistical methods. Data dimensionality will be adjusted for
different time horizons, for example for very and ultra-short term forecasting the algorithms will
perform on detailed data (minutes), since for STLF data will be aggregated hourly over locations or
consumers’ profiles. For LP we’ll use distributed data mining with multi-node processing based on

Self Organizing Maps, k-means and fuzzy c-Means. Thus, data regarding type of consumption, tariffs
schemas’ influence, injection of RES, EV usage is transformed into a valuable information for
aggregators/DSO. Optimization algorithms will be developed based on LR and MIP techniques to
formulate the complex dynamic optimization problems that include several constraints at each layer.
In Layer 1 LR will be applied to decouple the utility constraint and provide tractable sub-problems
formulated as MIP model. A hierarchical DR bidding framework in the day-ahead markets, which
integrates customer DR preferences & characteristics in the DSO’s market clearing process, will be
developed in Layer 3. The DSO applies MIP to solution of the proposed DR model in the day-ahead
market clearing problem.
From BG-Tier, some aggregated data is loaded into RB-Tier to allow integration with on premises
Energy Management Systems (EMS) already installed at aggregator/DSO. For example, the LP data
will be partitioned by locations and consumers’ profiles and stored in RB-Tier for integration with
EMS. The DW-Tier will be developed on top of BG and RB tiers and will be subject oriented towards
the business analyses regarding consumers’ behaviour and wholesale electricity markets. Analytic
solutions require data governance, data quality and stewardship that are absolutely critical and are
achieved only through the DW-Tier. At this layer we’ll developed a set of algorithms for analytical
purpose and key performance indicators (KPIs) reporting. The framework will offer scalable web-
services for targeted users. The aggregators/DSO will analyse data through an online portal available
as web-services in CC platform. For consumers, the web content will be customized based on their
preferences & requirements (monitoring/controlling/scheduling the appliances, real time billing
information, micro-generation and SD). A section of the portal will be developed as a simulator
similar to a computer game to motivate consumers to optimize their electricity consumption and to
increase their awareness towards energy efficiency.
4. Conclusion
The full liberalization of the markets will lead to competitive prices and services offered by
electricity suppliers including aggregators. Through business models, the prototype will allow
suppliers to configure their tariffs and to attract new consumers groups as well. This will bring a
significant benefit to prosumers, who will obtain several incentive options. They will play an active
role in DR, having the possibility to gain profit, apart from saving money.
There are several impacts the consortium find relevant and beneficial for society. The multi-layer
aggregator services encourage active participation of prosumers through differentiated tariffs between
locally and centrally produced power, and through a market, handle storage and DR. This enables
local markets to solve many of the issues regarding implementation of SG, especially around
incentives, business models and adoption. This active participation will allow consumers to reduce
their electricity bill maintaining their comfort. The solution opens up for new innovative business
models where companies can develop new products combining wholesale and local markets, and
energy services such as flexibility management. Again, this increases the economic benefits for
participants willing to take active roles in local energy markets.
In addition, by aggregating and increasing the share of local RES, the costs for upgrading regional
and national grid is lowered. By balancing the consumption and production in a local area, especially
during peak hours, less energy is imported from the centralized grid and thereby reducing the amount
of energy transported through the transmission grid. As a consequence, grid costs for end-users will
diminish.
The proposed solution will also contribute in reducing the prices by making a market for the local
energy produced, and thus increasing the incentives for buying local energy. In a functioning market,
the price difference between locally produced energy and wholesale energy due to transportation
costs, also creates incentive for consumers to become prosumers and sell energy to the local market.
Multi-layer aggregator concept needs new market polices, legislation and/or incentives schemes
for SG infrastructure. By demonstrating that the solution can, under the right circumstances, give the
desired effects, the pilot trials will demonstrate how such regulative bodies can be structured for the
desired impact for adoption of SG technologies.

5. References
[1] Final 10-year ETIP SNET R&I roadmap covering 2017-2026. Available online: https://etip-
snet.eu/pdf/Final_10_Year_ETIP-SNET_R&I_Roadmap.pdf .
[2] EC Energy Union Package 2015. Available online: http://eur-
lex.europa.eu/resource.html?uri=cellar:1bd46c90-bdd4-11e4-bbe1-
01aa75ed71a1.0001.03/DOC_1&format=PDF .
[3] Khodaei A., Shahidehpour M., Choi J. Optimal Hourly Scheduling of Community-Aggregated
Electricity Consumption, in JEET, 2013, Vol. 8, Issue 6, pp. 1251-1260; DOI:
10.5370/JEET.2013.8.6.1251.
[4] Logenthiran T., Srinivasan D., Shun Z. Demand Side Management in Smart Grid Using Heuristic
Optimization, in IEEE Transactions on Smart Grid , 2012, Vol. 3, Issue 3, pp. 1244-1252;
DOI: 10.1109/TSG.2012.2195686.
[5] M. Parvania, M. Fotuhi-Firuzabad, M. Shahidehpour, Optimal demand response aggregation in
wholesale electricity markets, IEEE TRANSACTIONS ON SMART GRID, 2013, Vol. 4, No. 4.,
pp. 1957-1965.
[6] M. Parvania, M. Fotuhi-Firuzabad, M. Shahidehpour, ISO’s Optimal Strategies for Scheduling
the Hourly Demand Response in Day-Ahead Markets IEEE TRANSACTIONS ON POWER
SYSTEMS, 2014, Vol. 29, No. 6., pp. 2636-2645.
[7] ENTSO-E R&D & Innovation Roadmap 2017–2026. Available online:
https://www.entsoe.eu/publications/research-and-development-reports/rd-
roadmap/Pages/default.aspx .

Acknowledgments. This paper presents some results of the research project: Informatics solutions
for optimizing the operation of photovoltaic power plants (OPTIM-PV), project code: PN-III- P2-2.1-
PTE-2016- 0032, 4PTE/06/10/2016, PNIII – PTE 2016.