Agile demand response
Citation for published version (APA):Babar, M. (2017). Agile demand response: a complementary approach to increase demand response volumes in an active distribution network. Technische Universiteit Eindhoven.
Document status and date: Published: 13/12/2017 Document Version:
Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.
• The final author version and the galley proof are versions of the publication after peer review.
• The final published version features the final layout of the paper including the volume, issue and page numbers.
Link to publication
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:
www.tue.nl/taverne
Take down policy
If you believe that this document breaches copyright please contact us at:
openaccess@tue.nl
providing details and we will investigate your claim.
Demand
Response
A Complementary Approach to increase
Demand Response Volumes in an Active
A G I L E D E M A N D R E S P O N S E
A Complementary Approach to increase Demand Response Volumes in an Active Distribution Network
PROEFSCHRIFT
ter verkrijging van de graad van doctor aan de Technische Universiteit Eindhoven, op gezag van de rector magnificus, prof.dr.ir. F.P.T. Baaijens, voor een
commissie aangewezen door het College voor Promoties in het openbaar te verdedigen op
woensdag 13 december 2017 om 16:00 uur.
door
m u h a m m a d b a b a r
promotiecommissie is als volgt:
Voorzitter: prof.dr.ir. P.G.M. Baltus Promotor: prof.dr. I.G. Kamphuis Copromotor 1: dr. H.P. Nguyen Copromotor 2: dr. V. Cuk
Leden: prof.dr.ir. J.A. La Poutr ´e (TU Delft) prof. dr. Z. Hanzelka (AGH UST)
prof.dr.ing. A.J.M. Pemen (TU Eindhoven) Adviseur: ir. M. Bongaerts (Alliander)
Het onderzoek of ontwerp dat in dit proefschrift wordt beschreven is uitgevoerd in overeenstemming met de TU/e Gedragscode Wetenschapsbeoefening.
sustainable energy services) research program. It is the EU funded fellowships for doctoral programmes. This research was performed in part at Smart Lab in the Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Science and Technology, Krakow, Poland. The project was partially supported by Alliander, which is one of the largest distribution network operators in the Netherlands.
Muhammad Babar: Agile Demand Response, A Complementary Approach to increase Demand Response Volumes in an Active Distribution Network, © December 2017, Eindhoven, The Netherlands.
All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic, mechanical, including photocopy, recording, or any information storage and retrieval system, without the prior written permission of the copyright owner.
ISBN: 978-94-028-0865-0
S U M M A R Y
In the roadmap to a successful energy transition, the EU considers Demand Response (DR) as one of the key enablers to reach emission reduction targets by more efficient operation of the electricity system. Now, the EU at its core sees that a future DR paradigm cannot and should not be a replacement of the existing DR but a complement to it. In other words, the paradigm would and should coexist with current DR strategies, thus adding functionalities by using an evolutionary path. This complementary approach necessitates an agility in DR paradigm that should provide solutions to improve the compatibility of different DR strategies and improve the involvement of customer. Therefore, this thesis defines the concept, design, and model of the Agile DR paradigm.
To be precise, the design of the Agile DR paradigm is defined as a Multi-agent System forming a three-layered hybrid architecture. The three layers are referred to application, coordination and access layers. Each layer consists of strategic processes to achieve one or more objectives, using a transactive control mechanism. In this way, agents within the same layer have peer-to-peer interactions, whereas agents interactions between the layers are confined to two signals. The first signal is known as a bid, whereas the second signal is known as a price signal. In order to standardize these signals for data abstraction and reduction in communication burden, this dissertation proposes the composite bidding rules. Moreover, the design supports continuous integration, which is a method that uses artificial intelligence to have DR automatically scheduled and dispatched. By characterizing the design in four prime agility enablers, the agile methodology is defined. In light of literature, the four prime agility enablers are the three-layered hybrid architecture, the transactive control, the composite bidding and the adaptation and learning.
Firstly, the dissertation attempts to estimate demand flexibility from the agents in the access layer. Agents in the coordinating layer, in association with an agent in the application layer, are responsible for monitoring the behavior of access agents. The Model-free Learning Techniques is proposed for strategic decision making at coordination layer. Secondly, the demand flexibility scheduling problem is solved as a multi-stage decision-making problem in the coordinating agents. The techniques use Reinforcement Learning approach to learn demand schedules, and then they control the access agents accordingly. However, the Pursuit Reinforcement Learning algorithm is proposed for reaching a relatively stable schedule during day-ahead as well as real-time operation. The algorithm is based on online learning that allows clustering of the data into several groups for the similar kind of access agents.
Finally, application agents are open to build either single or multiple strategies to improve the system responsiveness to the change. However, this dissertation only focuses on two strategies; (i) demand flexibility scheduling; and (ii) demand flexibility dispatch. These two strategies are distributed across two application agents, namely: the aggregator and network agent. The dissertation considers these strategies from two perspectives: a global perspective and a local perspective. A global perspective to DR is to maximize the net social welfare of the stakeholders. On the other hand, a local perspective is to mitigate network issues. The first strategy resides in the aggregator
the network agent and provides the local perspective to Agile DR that is to be used by network operators. The dissertation also shows how the two perspectives intertwine and are mutually supportive.
A co-simulation model of the architecture has been developed. The design provides strategic as well as architectural scalability with least complexity and a minimum of distributed intelligence, which are the prime attributes of an agile approach. The agile methodology is also found suitable for solving the demand flexibility scheduling problem within the Agile DR paradigm. Furthermore, the strategic needs for demand dispatch are adapted to the Netherlands grid settings for congestion management. In conclusion, this dissertation shows evidence that Agile DR is a complementary approach to traditional demand response. It has created mechanisms, agents, and learning techniques to respond quickly to customer needs and market changes while optimizing energy cost and network congestion. However, Agile DR is also capable of combining further strategies in a way that enhances social welfare of a community or emphasizes the power quality in a network.
S A M E N VAT T I N G
Op weg naar een succesvolle energietransitie ziet de EU (Europese Unie) Demand Respons (DR) als een van de belangrijkste instrumenten om de doelen voor uitstootreductie te bereiken door het elektriciteitsnetwerk effici ¨enter te bedrijven. Hierbij beschouwt de EU dat een toekomstig paradigma voor DR niet een vervanging voor de bestaande DR kan en moet zijn, maar temeer een uitbreiding hierop. Met andere woorden, dit paradigma zou naast de bestaande DR-strategie ¨en moeten bestaan en daaraan functionaliteit toevoegen door ontwikkeling van het bestaande model. Dit noodzaakt tot agility in het DR-paradigma, dat oplossingen aan moet reiken om de compatibiliteit van verschillende DR-strategie ¨en evenals de betrokkenheid van de eindgebruiker te verbeteren. Om dit te bewerkstelligen definieert deze thesis het concept, ontwerp en model van het Agile DR-paradigma.
Om precies te zijn is het ontwerp van het Agile DR-paradigma gedefinieerd als een Multi-agent system, dat een drie-laags hybride architectuur vormt. De drie lagen worden aangeduid als toepassings-, co ¨ordinatie- en toegangslaag. Elke laag bestaat uit strategische processen om een of meerdere doelen te bewerkstelligen, gebruikmakend van een transactive control mechanisme. Hierbij hebben agents binnen dezelfde laag peer-to-peer interactie, terwijl interacties van agents tussen de lagen beperkt zijn tot twee signalen. Het eerste signaal staat bekend als een bieding, terwijl het tweede signaal bekend staat als het prijssignaal. Om deze signalen te standaardiseren voor gegevensabstractie en om tot een reductie van de communicatiebelasting te komen, stelt dit proefschrift regels voor composite bidding voor. Daarnaast ondersteunt het ontwerp continuous integration, een methode die gebruikt maakt van kunstmatige intelligentie om DR automatisch te plannen en af te wikkelen. Door het ontwerp in de vier belangrijkste agility instrumenten te categoriseren, wordt de agility methode gedefinieerd. De vier belangrijkste agility instrumenten binnen de literatuur zijn: de drie-laags hybride architectuur, transactive control, composite bidding en kunstmatige intelligentie.
Ten eerste doet het proefschrift een poging om flexibiliteit in energievraag van de agents in de toegangslaag in te schatten. Agents in de co ¨ordinatielaag, gelieerd aan een agent in de toepassingslaag, zijn verantwoordelijk voor het monitoren van het gedrag van de toegangsagents. ‘Model-free Learning Techniques’ worden voorgesteld voor strategische besluitvorming in de co ¨ordinatielaag. Ten tweede wordt het planningsprobleem voor vraagflexibiliteit opgelost als een meertraps besluitvormingsprobleem in de co ¨ordinerende agents. Zij gebruiken de Reinforcement Learning benadering om planningen in energievraag te leren, waarop zij de betrokken toegangsagents aansturen in overeenstemming daarmee. Het “Pursuit Reinforcement Learning” algoritme wordt echter voorgesteld om een relatief stabiele planning te bereiken, zowel op voorhand voor de volgende dag alsmede voor real-time bedrijf. Het algoritme is gebaseerd op online learning, dat gegevensclustering in verschillende groepen toelaat voor dezelfde typen toegangsagents.
Ten derde, is het voor toepassingsagents mogelijk te werken aan zowel een enkele als aan meerdere doelstellingen om het reactievermogen van het systeem op verandering te
vraagflexibiliteit; en (ii) het afwikkelen van deze flexibiliteit. Deze twee doelstellingen zijn gedistribueerd over twee toepassingsagents, namelijk: de aggregator- en netwerkagent. Dit proefschrift beschouwt deze strategie ¨en vanuit twee perspectieven: een globaal perspectief en een lokaal perspectief. Het globale perspectief op DR gaat over het maximaliseren van de netto social welfare. Aan de andere kant, het lokale perspectief gaat over het verhelpen van netwerkproblemen. De eerste strategie bevindt zich in de aggregatoragent en levert het globale perspectief op Agile DR. De tweede strategie is geimplementeerd in de netwerkagent en levert het lokale perspectief op Agile DR om gebruikt te worden door netwerkbeheerders. Het proefschrift laat tevens zien hoe de twee perspectieven samenvlechten en elkaar onderling ondersteunen.
Een co-simulatiemodel van de architectuur is ontwikkeld. Het ontwerp voorziet in zowel een strategische als architectonische schaalbaarheid met de minste complexiteit en het laagste niveau van gedistribueerde intelligentie, wat de architecturele hoofdeigenschappen van een agile benadering zijn. De agile methode wordt eveneens geschikt bevonden om het planningsprobleem voor vraagflexibiliteit op te lossen binnen het Agile DR-paradigma. Voorts zijn de strategische vereisten voor vraagafwikkeling aangepast aan de Nederlandse netwerkinstellingen voor het verhelpen van overbelastingen.
Concluderend laat dit proefschrift het bewijs zien dat Agile DR een complementaire benadering is op traditionele DR. Agile DR bevat mechanismen, agents en leertechnieken om snel te reageren op klantbehoeften en marktveranderingen onder het minimaliseren van energiekosten en netwerkoverbelasting.Daarnaast is Agile DR eveneens in staat om verdere strategie ¨en te combineren op een manier die de social welfare bevordert van een gemeenschap of nadruk legt op de power quality in het netwerk.
C O N T E N T S
1 p r o l o g u e 1
1.1 Background and Motivations . . . 1
1.2 Demand Response in Europe Today . . . 1
1.3 The Agility Challenge . . . 5
1.4 Research Objective . . . 7
1.5 Thesis Scheme . . . 7
2 a g i l e d e m a n d r e s p o n s e 9 2.1 Introduction . . . 9
2.2 Agile Demand Response . . . 10
2.2.1 Agile DR Conceptual Model . . . 10
2.2.1.1 Agility Drivers . . . 11
2.2.1.2 Agility Enablers . . . 11
2.2.1.3 Agility Outcomes . . . 13
2.3 Agile Methodology . . . 13
2.4 Transactive Control Mechanism . . . 14
2.4.1 Attributes of Transactive Control . . . 14
2.5 Three Layered Hybrid Architecture . . . 17
2.5.1 Micro-Modeling . . . 18
2.6 Bidding . . . 20
2.6.1 Simple Bidding . . . 22
2.6.2 Composite Bidding . . . 22
2.6.3 Composite Bidding Rules . . . 23
2.7 Adaptation and Learning . . . 25
2.7.1 Centralized learning . . . 25 2.7.2 Distributed learning . . . 26 2.8 Summary . . . 26 3 d e m a n d f l e x i b i l i t y 29 3.1 Introduction . . . 29 3.2 Demand Elasticity . . . 29 3.3 Learning-based System . . . 31 3.3.1 Functional Profiles . . . 31
3.3.2 Energy System Agent . . . 34
3.3.3 Energy Logger . . . 34
3.3.4 Elasticity Agent . . . 34
3.3.5 Appliance Agent . . . 35
3.4 Demand Elasticity Estimation . . . 35
3.4.1 State Space and Action Space . . . 36
3.4.2 The Objective Function . . . 36
3.4.3 Action Selection . . . 36
3.4.4 Simulation of Demand Elasticity by using the Algorithm . . . 37
3.5 Experimentation . . . 39
3.5.1 Software Design . . . 39
3.5.2 Hardware Design . . . 40 3.5.3 Performance Metrics . . . 41 3.5.4 Results . . . 42 3.6 Summary . . . 43 4 d e m a n d f l e x i b i l i t y s c h e d u l e 45 4.1 Introduction . . . 45
4.2 Agent-oriented Distributed Scheduling System . . . 45
4.2.1 Aggregator Agent . . . 46
4.2.2 Domotic Agent . . . 47
4.2.2.1 Energy Cost Function . . . 48
4.2.2.2 Objective function . . . 48
4.2.2.3 Principle of the Learning . . . 49
4.2.3 Allocation Agent . . . 50
4.2.3.1 State Space and Action Space . . . 50
4.2.3.2 State Transition Function . . . 51
4.2.3.3 Reward Function . . . 51
4.2.3.4 Convergence and Optimality . . . 53
4.2.4 Grouping Agent . . . 54
4.3 Illustrative Studies . . . 56
4.3.1 Heterogeneous Atomic Configuration . . . 57
4.3.2 Experimentation for Q-Iteration . . . 57
4.3.3 Convergence Simulation . . . 58
4.3.3.1 Offline Learning . . . 59
4.3.3.2 Online Learning . . . 59
4.3.4 Performance Results . . . 60
4.3.5 Homogeneous Atomic Configuration . . . 61
4.3.6 Homogeneous In-atomic Configuration . . . 62
4.4 Summary . . . 62 5 d e m a n d f l e x i b i l i t y d i s pat c h 65 5.1 Introduction . . . 65 5.2 Agile Net . . . 65 5.2.1 Network agent . . . 66 5.2.2 Host agent . . . 66 5.2.3 Percept agent . . . 67 5.2.3.1 Reward Function . . . 67
5.2.3.2 A responsive virtual environment . . . 68
5.2.3.3 State transition . . . 69
5.2.4 Dispatch agent . . . 70
5.2.4.1 Markov Decision Process . . . 70
5.2.4.2 Tile Coding . . . 71
5.3 Experiment Design . . . 73
5.3.1 Configuration . . . 73
5.3.2 Methodology . . . 74
5.3.3 Overall Performance . . . 74
5.3.4 Congestion Management with Single Aggregator . . . 75
c o n t e n t s xi
5.4 Summary . . . 77
6 e p i l o g u e 79 6.1 Conclusions . . . 79
6.2 Contributions . . . 80
6.2.1 Three-Layered Hybrid Architecture . . . 80
6.2.2 Micro-Modeling . . . 80
6.2.3 Composite Bidding . . . 81
6.2.4 Demand Elasticity . . . 81
6.2.5 The Global Perspective . . . 82
6.2.6 The Local Perspective . . . 82
6.2.7 The Agile Perspective . . . 82
6.3 Future Recommendations . . . 83 a a p p e n d i x 87 b a p p e n d i x 89 c a p p e n d i x 91 d a p p e n d i x 95 b i b l i o g r a p h y 100
Figure 1.1 Radar maps showing the DR advancements across 14 Member States
of the EU in the light of the elemental criteria for DR review. . . 3
Figure 1.2 The Agility Challenge. . . 5
Figure 1.3 Agility in a DR paradigm. . . 6
Figure 1.4 The Thesis Schema. . . 8
Figure 2.1 Conceptual model of Agile DR. . . 13
Figure 2.2 Multi-Agent System. . . 18
Figure 2.3 Three-Layered Hybrid Architecture for the agile methodology. . . 19
Figure 2.4 UML Class Diagram as a conceptual model of an Agent-oriented Agile Methodology. . . 20
Figure 2.5 UML Sequence Diagram of the agile methodology. . . 21
Figure 3.1 Knowledge-based ontology representing functional agents within the access layer and their peer-to-peer interactions. . . 32
Figure 3.2 Functional profiles of agent-oriented functional components of an appliance agent. . . 33
Figure 3.3 The absolute error between the expected price and the real price. . . . 37
Figure 3.4 Probability density function along with the vector of actual price signals of the last simulation day. . . 38
Figure 3.5 The difference between the numerically calculated elasticity and the elasticity estimated by the proposed algorithm. . . 39
Figure 3.6 A hybrid three-layered organization of agents for agile DR. . . 40
Figure 3.7 Experimental setup by using the hardware-in-loop technique. . . 41
Figure 3.8 (Upper graphs) bids generated by appliance agents, and (Lower graphs) turn ON timings during a demonstration in response to price signals. . . 42
Figure 3.9 The average curves of the aggregated demand curves at the aggregator for the complete duration of the experiment. . . 42
Figure 4.1 Knowledge-based ontology, representing the agent-oriented distributed scheduling system using peer-to-peer interactions. . . 46
Figure 4.2 Slice through the offline Q-learning after 100 days of demand flexibility scheduling while considering; (right) the simple bidding and (left) the composite bidding. . . 57
Figure 4.3 Slice through the online Q-learning after 100 days of demand flexibility scheduling while considering; (right) the simple bidding and (left) the composite bidding. . . 58
Figure 4.4 Resource dilation factor calculated over 100 days of simulation. . . 59
Figure 4.5 Convergence of reward function for an initial day while learning process . . . 60
Figure 4.6 Performance of a various number of domotic agents in the self-elastic as well as the cross-elastic scenarios. . . 61
Figure 5.1 Knowledge-based ontology, representing Agile Net using peer-to-peer interactions. . . 66
List of Figures xiii
Figure 5.2 The Tile Coding based Q-table. . . 71 Figure 5.3 Imaging the Agile DR paradigm to Bronsbergen low voltage network. 72 Figure 5.4 Average learning duration in seconds under various reconfiguration. . 76 Figure 5.5 Performance of Agile Net in multiple aggregators scenario. . . 77 Figure A.1 A schematic diagram of an appliance agent with CS5460 IC and the
Raspberry Pi microcontroller. . . 87 Figure A.2 Finite state machine of the appliance agent. . . 88 Figure B.1 State space transition for the appliance agent in three different paths. 89 Figure C.1 Offline algorithm for Homogeneous atomic configuration, chosen
action for group Ga representing (αd,Ga, βd,Ga,ωd,Ga)=(1, 10, 3). . . . 91 Figure C.2 Online algorithm for Homogeneous atomic configuration, chosen
action for group Ga representing (αd,Ga, βd,Ga,ωd,Ga)=(1, 10, 3). . . . 92 Figure C.3 Total cost for unscheduled and scheduled demand for Homogeneous
atomic configuration. . . 92 Figure C.4 Total cost for unscheduled and scheduled demand for Homogeneous
atomic configuration. . . 93 Figure D.1 Real topology of the LV network in Bronsbergen . . . 95
Table 1.1 Elemental Criteria for DR review. . . 2
Table 2.1 Demand Response from an agile perspective. . . 10
Table 2.2 Summary of Agile DR strategies, existing practices and references . . 12
Table 2.3 Descriptions of control mechanisms in view of system organizations verses DR control strategies. . . 15
Table 3.1 Average training time . . . 41
Table 4.1 State Space . . . 50
Table 4.2 State Action Space . . . 51
Table 4.3 The total average cost per allocation decision in the homogeneous atomic configuration . . . 61
Table 4.4 Total average cost per allocation decision in homogeneous in-atomic configuration . . . 62
Table 5.1 Overall Performance Summary . . . 74
Table 6.1 A side-by-side comparison of the ideal perspective of Agility and the compromised conditions at the Agile DR paradigm. . . 84
Table C.1 The average cost during the day for hourly intervals. . . 91
Table C.2 Schedule of five in-atomic buffer agents. . . 93
N O M E N C L AT U R E
The main notation used throughout the dissertation is stated below for a quick reference. Other symbols are defined as required throughout the dissertation.
va r i a b l e s
k Index refers to the current time interval.
K The total number of time intervals per periodic time period (i.e. a day). Kd,a The operating window of the appliance agent-a of the domotic agent-d.
k0 A time interval other than the current time interval.
A The number of appliance agents associated with the domotic agent. D The number of domotic agents.
Ed,a The total energy in kWh of the appliance agent-a of the domotic agent-d in a day.
εkk The self-elasticity of demand. εkk0 The cross-elasticity of demand.
lkd The total demand of the domotic agent-d during the time interval k. ld The total demand curve.
bd The total non-flexible demand vector (i.e. the base-load).
xd The total flexible demand vector.
δ An action chosen by an agent randomly during an iterative learning process.
∗
δ The optimal action obtained after an iterative learning process. Q(s, δ) The Q-value function for a state-action pair.
r The reward function calculated during an iterative learning process for the demand dispatch problem.
g The reward function calculated during an iterative learning process for the demand scheduling problem.
I The total number of iterations. N The total number of steps. η Step size of the learning.
s e t s a n d pa r a m e t e r s
Ad A set of appliance agents associated with the domotic agent-d.
D A set of domotic agents associated with the aggregator.
λkd,a A simple bid vector for the appliance agent-a of the domotic agent-d during the time interval k.
ηd,a A composite bid vector for the appliance agent-a of the domotic agent-d. αd,a The start time interval of the appliance agent-a of the domotic agent-d. βd,a The stop time interval of the appliance agent-a of the domotic agent-d. ωd,a The total operating duration of the appliance agent-a of the domotic agent-d. γkd,a The power in kW of the appliance agent-a of the domotic agent-d during
the time interval k.
γd,a The demand profile of the appliance agent-a of the domotic agent-d during the time interval k.
θd,a A boolean parameter that represents the nature of the appliance agent-a as being atomic or in-atomic.
Λd,a A vector of the estimated simple bids of the appliance agent-a for a day. Γk The price signal during the time interval k.
P(δ) The probability of taking an action δ.
s The state of an agent during an iterative learning process. Sa The state space of the appliance agent-a.
φk The normalized value of congestion at a transformer. ϕk The probability of congestion occurrence.
A C R O N Y M S
API Application Programming Interface DR Demand Response
DSO Distributed System Operator EU European Union
EEU European Energy Union EV Electric Vehicle
ICT Information and Communication Technology IoT Internet of Things
LV Low Voltage
MAS Multi-Agent System MDP Markov Decision Process PEV Plug-in Electric Vehicle
PDF Probability Distribution Function PRP Program Responsible Party RES Renewable Energy Resources TCM Transactive Control Mechanism TE Transactive Energy
UML Unified Modeling Language
1
P R O L O G U E
1.1 b a c k g r o u n d a n d m o t i vat i o n s
In November 2016, the EU Commission has published its Clean Energy for All Europeans package, more commonly referred to as the Winter Package [3]. The package consists of numerous legislative proposals and new communications, mainly addressing the challenges and issues to achieve competitive, transparent and responsive internal market for electricity. Thus, it further increases the possibility of the implementation of the European Energy Union (EEU) [4].
There are many divisive challenges in the winter package. However, the EU Commission has unanimously accepted that Europe should increase flexibility in its energy system to achieve the target of 27% of Renewable Energy Resources (RES). In the wider context, the package has made three majors proposals to Directive 2009/72/EC. [5]. Firstly, the technical and regulatory hurdles that prevent Demand Response (DR) service providers (i.e., aggregators) to participate in all segments of the market should be removed. Secondly, Distribution System Operators (DSOs) should actively provide services to ensure flexibility, storage and recharging points for electric vehicles. Above all, the rights of consumers and/or prosumers must be expanded, thus enabling explicit provision of flexibility by voluntarily changing their usual electricity consumption in reaction to price signals or specific requests.
In the wider context of DR, the package reforms Directive 2012/27/EU on Energy Efficiency [6]. The reforms in the articles are the outcomes of findings obtained from the realization articles 15.4 and 15.8 of the legislation throughout the EU over the last decade. It has been found that the phenomenon of DR is now evolutionary rather than revolutionary. Therefore, the focus of the current reforms is to provide fundamental rights and rules that can evolve and enable consumers, technologies, and strategies to fulfill the EU 2030 target for 30% of energy savings. The proposal will drive the EU Member States to develop and implement new ideas and trends in their electricity markets and power systems to hit the 2030 energy target on time.
1.2 d e m a n d r e s p o n s e i n e u r o p e t o d ay
Herein, a qualitative analysis is conducted to review in-depth the data that could be quantified to map the current state of DR across the EU. The outcomes of the qualitative analysis are twofold. Firstly, the qualitative analysis provides a state of art developments in the area of DR across the EU. Secondly, and most importantly, it highlights a solid research niche, which specifies the need for a complementary approach to increase DR volumes in an active distribution network.
This chapter summarizes the key contributions made in [1,2].
a c r o n y m d e f i n i t i o n d e s c r i p t i o n EX Explicit The overall existence of Explicit DR.
AG Aggregation Participation and engagement of aggregator independently or via mediator in the electricity markets.
PR Program
Requirements
The requirements of the different DR products and/or programs.
MV Measurement and
Verification
The MV is the quantification and validation of the dispatched demand.
PP Payments and
Penalties
Payment structures and penalties for enabling DR. IM Implicit The overall existence of Implicit DR.
SM Smart Metering Smart metering penetration.
SP Service Provider Extensive availability of service providers. This will allow customers to choose SP according to its satisfaction.
IS Innovative Services Innovative services available commercially and economically attractive to customers (like energy management, IoT, etc.).
DP Dynamic Pricing Existing pricing methodologies that can implicitly reflect customer’s demand in electricity markets. Table 1.1: Elemental Criteria for DR review.
Ten elemental criteria, as shown in Table 1.1, are considered as outset to perform qualitative analysis for DR status across a region. The study uses the scoring system for DR review. The system grades each elemental criterion on the scale of 0 to 5. Grade 5 is seen as the most effective score of linking evidence-quality analysis to a criterion. The investigation is conducted on collected data and information accumulated from the recent reports on Smart Grid and DR published by the EU Joint Research Centre (JCR) on Smart Electricity Systems and Interoperability [7, 8], and Smart Energy Demand Coalition (SEDC) [9, 10]. JCR provides the most recent database of smart grid projects across the EU [7]. The current smart grid outlook in [8] includes the total of 459 smart grid projects since 2010. SEDC publishes information on the progress of the EU regarding DR [9]. Ref. [10] shows the recent development of the EU in enabling explicit DR.
The analysis results are summarized into radar graphs, as shown in Figure 1.1. The radial axes of each graph represent the relationship between each elemental criterion versus five qualitative grades. In the radar graphs, each grade of the polygon represents a quantity of the DR state examined concerning each elemental criterion. The results on radar graphs map a quick visual comparison of the status of DR. It is observed that the efforts of the EU as a whole appear to be favorable in DR enablement. The overall study, performed in [1], revealed four areas for further improvements:
1.2 demand response in europe today 3
r e g u l at o r y f r a m e w o r k Although the regulatory framework for the enablement of DR is more advanced than it was a couple of years ago, it is still fragmented. More work needs to be done to accelerate the promotion of Demand Response across all Member States. However, most of the concerns related to the EU Internal Energy Market has already been addressed within the EU winter package.
i n d e p e n d e n t a g g r e g at o r s The majority of the EU Member States do not yet acknowledge the role of independent Demand Response aggregators, or they require aggregators to include bilateral contracts with retailers/program responsible parties (PRPs) - whose business is often competing - to sell consumers’ demand flexibility.
e l e c t r i c i t y r e ta i l p r i c i n g Significant progress has been made by most of the Member States in giving DERs and DR access to the retail pricing environment. In particular, balancing electricity markets in few Member States have been opened for pilot projects. However, the rest of the Member States have opened their wholesale market for DR.
d r p r o g r a m s To fully enable DR across Europe, the program needs to ensure that the mechanism is in the best interest of consumers and other market actors, and fit for the modern electricity market. Keeping previous DR programs in place does not encourage the use of market-based demand flexibility today. Hence, it is imperative now more than ever that Member States should enable DR services through market competition. Therefore, new integrated DR programs required to be developed and proposed.
Within these areas, Member States are classified into three groups depending on their DR developments, namely; Nascent, Intermediate and Mature. In the Nascent group, Italy, Poland, Slovenia and Spain have to take serious measures in almost all areas. Germany, the Nordics, the Netherlands, and Austria are in the Intermediate group. This group has shown significant progress in regulatory framework and electricity retail pricing. However, there are no integrated DR programs that have been implemented, the exception to a few
Austria Belgium Denmark Finland France Germany
UK Ireland Italy Netherlands
Norway Poland Slovenia Spain
EX AG PR MV PP IM DP IS SP SM Europe
Figure 1.1: Radar maps showing the DR advancements across 14 Member States of the EU in the light of the elemental criteria for DR review.
pilots. Although Belgium, France, Ireland and the UK are grouped into Mature, there is still room for development in DR programs that able to source flexibility for optimized local system operations.
Although the focus of the DR review herein is limited to the EU Member States, the process of DR review can be applied to investigate the current state-of-the-art in any country of the world. The analysis, in general, helps in finding the grey areas in the development of DR, which need further research for design and development. From the study, it is found that DR is not a new concept or ideology in the EU. Nevertheless, its complete enablement is yet a challenge. The enablement of DR consists of two phenomena: (i) the integration of DR to electricity markets and (ii) the customer empowerment.
Former phenomenon is revolutionary because it is intended to outline the primary legal conditions. It refers to policies, rules, and regulation that align the economic interested of the stakeholders (i.e., consumers, retailers, DSOs, PRPs) with DR objectives. In this regard, as discussed, the Member States have already taken measures for a comfortable and smooth uptake of DR into their power systems. The EU commission is recently taking steps to ensure significant penetration of DR in the EU by 2030 [4]. As observed from Figure 1.1, the measures in the most of the Member States are highly skewed towards the development of implicit DR. Since, implicit DR provides time-varying electricity or network tariffs or both to customers. On the other hand, the customers react to the price change according to their needs and availability. Therefore, implicit DR permits a more considerable demand to be marketed at the lower price during off-peak hours. Consequently, both the participating as well as the non-participating customers unquestionably gain from implicit DR. Hence, the distribution of DR benefits across the participating customers is indeterminate due to the absence of an exact magnitude of the shifts in demand by the customers. However, implicit DR scheduling is found to be an essential strategy from the point of maximizing social welfare.
The latter phenomenon is evolutionary because the customer empowerment is a process that cannot be guaranteed by legal policies and contracts that are negotiated. Diffusion of the advanced ICT and IoT into the power system has created new opportunities for the customer enabling. The customer enabling is the way in which DR paradigm deals with the customer retention. There have been many different ways to incentivize participating customers in DR while satisfying their needs. These methods are referred to as Explicit DR. It provides services to transact demand flexibility in electricity markets, whereas the customers are incentivized for their flexibility. In short, explicit DR empowers the customers by providing them opportunities to dispatch their demand flexibility on a real-time basis in a market.
Customers are more empowered today than ever before. Moreover, the advanced ICT and IoT are accelerating the trend toward higher customer empowerment. Therefore, it is indeed a challenge to design and develop DR paradigms that focus on the customer first, specifically each participating customer in real time, to build a long-lasting mutual trust between the other stakeholders.
1.3 the agility challenge 5
Stage 1 Stage 2 Stage 3
Contract Opportunism Trust
Increasing Collaboration C S T C S T C S T Agile
C-Customer enabling S-Strategic processes T-Technologies Mutual benefits from complementary competencies Area of trust from an agile prespective
Figure 1.2: The Agility Challenge.
1.3 t h e a g i l i t y c h a l l e n g e
In an agile manufacturing, the trust is mutually beneficial collaborations created through complementary competences of the customer enabling, the strategic processing and the technologies, as shown in Figure 1.2. The challenge to build the trust in the agile manufacturing is known as the agility challenge1
. Therein, the customer enabling refers to the role of customer and its knowledge in the strategic processes. The strategic processes influence how manufacturers and customers may be leveraged. Customers making the purchasing decisions are considered for knowledge about short-term changes in market needs, while manufacturer works directly with the products and continuously improved development processes for innovation. However, continuous integration in an agile manufacturing is a practice that requires workers to integrate their work frequently. Integration is supported by advanced technologies, which refers to a combination of Communication and Information Technologies (ICT), learning techniques, data analytics and particular methods that should be used in the production. In this way, herein the technologies are about identifying the practices for acquiring, distributing, and using customer and developer known for the innovative development and/or continuous improvement.
In the DR development process, the challenge to build trust from an agile perspective can be viewed as the agility challenge. The customer enabling refers to the role of customer’s preferences and the primary process parameters of its appliances. Herein, a customer can be an industrial, commercial or domestic end-user. One of the customers may also be a DSO when it wants to utilize demand flexibility for network management. The strategic processes influence how DR service provider (i.e., an aggregator2
) and customers may be leveraged. The technologies here do not merely refer to the availability of the advanced ICT and IoT. However, they also support continuous integration. Continuous integration requires methodology, which is a particular design that uses
1 Agility is a continuous development process that has been defined as an ability of a developed system to grow
in a competitive market of continuous and unanticipated change. The system should be capable to respond to rapidly changing markets, which are driven by customer feedback.
2 “Aggregator” is an entity which independently or via PRP trades demand flexibility to electricity markets
complementary methods customer enabling
strategic
processes technologyadvanced
customer involment embedded design user’s flexibility strengthen collaboration ICT and IoT smart contollers scalable solutions consider customer needs adaptable to market changes real time control optimal resource schedule forecating behavior
Figure 1.3: Agility in a DR paradigm.
artificial intelligence to have DR automatically scheduled and dispatched. In this way, the technologies support in the implementation of DR services and the accomplishment of objectives. Hence, the paradigm continuously improves its responsiveness to changes in markets.
Babar et al. in [2] discussed in details the agility challenge by pairwise comparing the most influential factors of the existing DR paradigm based on transactive energy, employed multi-agent technology and customer-oriented energy flexibility platform and interface. By pairwise comparing the most prominent elements, as shown inFigure 1.3, it was visibly implied that the mutual region of the three collaborators provides the global effectiveness of agility in DR developments. Thus, all the varieties of DR developments are required simultaneously for reinforcing complementary approach. Therefore, to design and develop the Agile DR paradigm as a complementary approach, it is indispensable to select and focus on the existing DR strategies as well as acquire and realize the core competence in the agile methodology.
In short, Figure 1.3shows that agility can be achieved in a DR paradigm by increasing the collaboration between the customer enabling, the strategic processes and the technologies. The EU Commission’s legislative proposals within the winter package are ready to make transitions from the existing DR paradigm to the stage of agility [3]. It can be inferred from the radar graphs that the Member States in the intermediate and mature group have already reached the stage of opportunism. Because they have already provided industrial, commercial and domestic customers with opportunities to use ICT effectively and explore DR services provided by an aggregator [2].
1.4 research objective 7
1.4 r e s e a r c h o b j e c t i v e
The very first step in the challenge of agility is the development of concept, design, and methods for Agile DR. The research question therefore in this thesis is formulated as:
To provide a concept and content of Agile DR, developed through complementary competences of the existing DR methods and strategies in an electricity market environment.
To assess the potential benefits of agility for DR, Agile DR is briefly defined as a complementary approach that enhances DR volumes in an active distribution network. To answer the research objective, this thesis formulates and addresses the following sub-questions in the subsequent chapters:
1. What are the existing DR paradigms? How can a DR paradigm be conceptualized from an agile perspective? (Chapter 1)
2. How can the agility practices in DR be selected, assessed, and consolidated in Agile DR paradigm? What is the most suitable methodology for the Agile DR paradigm? (Chapter 2)
3. How can Agile DR benefit the stakeholders especially regarding cost saving as well as increased responsiveness to the changes in a market environment? (Chapter 2) 4. What are the challenges with current automation systems and specific requirement
for Agile DR from the aspect of physical development? (Chapter 3)
5. What is the economic potential of demand flexibility schedule in electricity retail pricing? How could the demand scheduling problem fit into Agile DR from the global perspective of strategic implementation? (Chapter 4)
6. What is the demand flexibility dispatch? How can it be seamlessly integrated into the congestion management to improve market responsiveness? (Chapter 5)
1.5 t h e s i s s c h e m e
The thesis is structured to address the formulated research questions and is divided into four chapters. Each chapter constitutes a comprehensive summary of the appended papers and clarifies the significant contributions therein. Each chapter starts with an introduction of the addressed topics and provides the synthesis of conclusions by combining analysis and results from several of the published, accepted and under review papers.
In The List of Publications, a series of papers, published, accepted and under review in peer-reviewed international conferences and journals, is presented as an individual chapter of the thesis. The thesis scheme is graphically depicted inFigure 1.4.
In brief, Section 2.3 provides the necessary contextual model for Agile DR in light of agility drivers, enablers and outcomes. Later, it focuses on the evaluation of the agile methodology, which is defined in the view of four prime agility enablers, which are: the three-layered hybrid architecture, the transactive control, the composite bidding and the adaptation and learning. Chapter 3 attempted to estimate demand flexibility using the
Chapter 2 Agile Methodology Chapter 3 Demand Flexibility Chapter 4 Demand Flexibility Schedule Chapter 5 Demand Flexibility Dispatch C S T
Agile Demand Response
Figure 1.4: The Thesis Schema.
concept of demand elasticity. A methodology is used to micro-model appliance agents. Moreover, Chapter 4 and Chapter 5 develop Agile DR strategies from two different perspectives; namely demand flexibility schedule and dispatch, respectively. For former perspective, the practicality of the methodology is demonstrated by implementing demand scheduling system. However, for latter perspectives, the practicality of the methodology is shown by solving demand dispatch problem for network congestion.
2
A G I L E D E M A N D R E S P O N S E
2.1 i n t r o d u c t i o n
The main driving force behind agility is a change in electricity markets during the last decades across the world. Especially in Europe, the unbundling and the liberalization of electricity markets and advances in Smart Grid Technologies have already revolutionized power systems of many member states. As discussed in Chapter 1, the EU is moving ahead to empower customers as well as increase competitiveness and improve responsiveness to the change in electricity markets. Despite the fact that the customer empowerment, improved responsiveness, and process adaptability are prime ingredients of agility, little empirical evidence exists in the literature that formulates the concept of agility. Agility, as a concept in dynamic pricing strategy, was coined by Paul A. Centolella at Analysis Group, an economic and strategy consulting firm, in 2012 [12]. In [12], Centolella presented an integrated strategy that includes dynamic pricing and engaging intelligent devices to perform DR while satisfying customer’s needs. He concluded that the agility in DR could play a significant role in the power industry’s future. Agility can be introduced as a part of DR services designed in a way that it can meet consumers’ needs more efficiently and reliably.
Since being agile is different than being flexible, so the study related to agility in DR is in the early stages, and further understanding and research is required. Therefore, this dissertation provides further understanding and research for the design and development that addresses how Agile DR can be achieved with a clarity of purpose, focus, and goals. This chapter essentially reviews possible interpretations of the agility in DR and proposes a comprehensive definition. Later in the chapter, the concept of Agile DR is embraced in the light of the literature. Section 2.2.1 presents a conceptual model for Agile DR, thus identifying three basic elements: agility drivers (environment), agility enablers (agility practices) and agility outcomes. Thus the integrated system of control methods and mechanisms used in the study of the Agile DR paradigm is referred to as the agile methodology. For the agile methodology, Transactive Control Mechanism (TCM) has been considered as a control method of Agile DR, as presented in Section 2.4. Section 2.5 explores hybrid structuring of multi-agents and Section 2.5.1 exploits it by describing micro-models of applied agents. Section 2.6 identifies the significance of energy bidding in the agile methodology and presents rules for the composite bidding strategy instead of simple. Furthermore, the principle of distributed learning as adaptive controls is presented inSection 2.7.
This chapter summarizes the key contributions made by M. Babar in [11].
a g i l e p e r s p e c t i v e Philosophy Learning-based method
Goal Adaptation, flexibility, responsiveness Environment Turbulent, difficult to forecast
Type of learning Double-loop/generative Problem-solving
process
Learning through exploration and introspection Constantly reviewing the activities
Methodology Evolutionary
Iterative and exploratory
Integrated formulation and implementation Demand-driven
Key Collaborative demand dispatch characteristics Open, scalable architecture
Encourages exploration and creativity Aggregator is facilitator
Design and implementation are inseparable and evolve iteratively Table 2.1: Demand Response from an agile perspective.
2.2 a g i l e d e m a n d r e s p o n s e
In principle, Agile DR addresses the challenge to survive in an unpredictable market situation while considering the customer’s value propositions and their interactions (either implicit and/or explicit) with other stakeholders.Table 2.1summarizes how a DR paradigm can be perceived as Agile DR. Table 6.1 in Chapter 6 indicates that this dissertation has achieved all these characteristics to some extent. Babar et al. have already presented detailed insights into agility and the paradigm definitions in [11].
Although Agile DR owes a lot to advances in ICT as well as the adoption of both implicit and explicit paradigms of DR, it is more than a hybrid construct of ICT and any previous paradigm of DR. It advocates a holistic, rather than a sub-optimal, approach to DR. Therefore, Agile DR is comprehensively defined as “a paradigm that can quickly satisfy customers’ value propositions; can introduce new demand-side services frequently promptly, and can make strategic decisions speedily.”
2.2.1 Agile DR Conceptual Model
Agile DR is conceptualized in this thesis as a model that integrates technology, customer empowerment, and power systems by creating a control mechanism using advanced ICT, granting demand flexibility, efficiently scheduling demand and its real-time dispatch.
2.2 agile demand response 11
Together with mutualizing all the mentioned strategic features, it makes sure to respond deliberately, efficiently, and in a coordinated fashion to changes in an electricity market.
Based on a thorough review of the literature and the analysis of several existing practices, as shown in Table 2.2, the conceptual model of Agile DR is encapsulated in three basic elements: agility drivers, agility enablers, and agility outcomes.
2.2.1.1 Agility Drivers
An increase in volatility in electricity retail pricing is subject to discussion. Brown et al. in [40] discussed in detail about the existing challenges in electricity retail pricing. Together with the challenges associated with market volatility due to retail pricing, authors in [40] reviewed policies that indirectly address market volatility. Such policies may include aging of network infrastructure, retiring generation capacity, reduction in carbon emission and subsidies to promote investment in renewables, etc.
From the discussion, it is inferred that the retail price volatility can be considered to be the most challenging environment for endurance. Aggregators that can operate successfully in such situations should show high levels of agility because they need to adapt to:
• a highly responsive market with unpredictable changes;
• a highly competitive environment with one or many critical and scarce energy resources;
• close collaboration with other stakeholders (suppliers, distributors, and customers); and
• a high diversity in customers (domestic, business, and industrial consumer and/or prosumer).
There is competition among retailers in the electricity market of the EU. Moreover, in the most of the Member States, retailers are providing dynamic tariff over a day-ahead as well as an hourly basis. It implies that the prime agility driver, i.e., the retail pricing is already existed to drive the design and development of Agile DR.
2.2.1.2 Agility Enablers
Based on the list of research and development on DR, as shown in Table 2.2, agility enablers can be grouped into five strategic areas, namely: technologies, integration, systematization, knowledge management, and customer empowerment.
The agile methodology must reflect the key enablers to design and develop the complementary nature of Agile DR in context of existing enablers. In this thesis, four prime agility enablers are considered for the development of the methodology. As shown in Section 2.3, the four enablers are hybrid architecture, transactive control, composite bidding and adaptation, and learning. Later, this chapter explains the four enables in details. In short, the agile methodology is not a general method but rather a specific set of methods to Agile DR based on Multi-agent System (MAS) by using a TCM.
s t r a t e g i c a r e a a g i l e p r a c t i c e s r e f . T echnologies Distributed automation and contr ol [13 , 14 ] Systematic implementation and integration of smart design, de vices and Aggr egating and matching [15 ] technologies. ICT Infrastructur e [16 , 17 ] Monitoring and Diagnosis [18 ] Integration Demand Response Exchange [19 ] Practices relating to the de v elopment of mechanisms for integrating and T ransactiv e Contr ol [20 , 21 ] coor dinating the v alue chain, based on operations amongst DR participants Demand flexibility market [22 , 23 ] and other stakeholders. portfolio management [24 , 25 ] Systematization Frame w ork [26 , 27 ] Practices to de v elop ne w pr oducts and/or pr ocess leading to standar dized or T ool de v elopment [28 , 29 ] systematized engineering. Standar d de v elopment [30 , 31 ] Knowledge Management Global access to data and infor mation [32 , 33 ] Practices relating to lear ning-based systems and lear ning. lear ning or ganization [34 , 35 ] Distributed lear ning [36 ] Deep Lear ning [37 , 38 ] Customer empowerment Inter net of things [39 ] T able 2 .2 : Summar y of Agile DR strategies, existing practices and refer ences
2.3 agile methodology 13
2.2.1.3 Agility Outcomes
Complementing the existing agility enablers/practices will lead to agility outcomes, that is the development of Agile DR. Therefore, the agility accelerates DR strengths through the development of more significant capabilities in different objectives (including energy cost, power quality, demand flexibility, demand dispatch, and market environment).
Agile Methodology
Composite Bidding
Hybrid Architecture
Adaptation andLearning
Transactiv e Control Demand Flexibility Demand Flexibility Schedule Demand Flexibility Dispatch Volatile Retail Pricing DR Benefits -Energy Cost -Flexibility -Quality -Dispatch -Schedule Drivers Outcomes
Agile DR
EnablerFigure 2.1: Conceptual model of Agile DR.
2.3 a g i l e m e t h o d o l o g y
From the previous discussion, it can be inferred that despite the advances in retail pricing, limited attention has been paid to develop a paradigm that facilitates both demand flexibility schedule and dispatch. A comprehensive and systematic methodology is required to distribute DR benefits across the participating customers fairly. An agile methodology, which is proposed herein, is expected to distribute DR benefits fairly by enabling agility throughout all market players. In this method, all market players that include aggregator, retailers, DSO, PRPs and especially the customers can achieve their objectives actively as well as transactively. A combination of the agility of the aggregator and other market players allows responsive behaviors in turbulent and volatile retail pricing. For this reason, the scope of this chapter is the formulation and analysis of the agile methodology. The methodology is based in Agent-based Transactive Control Mechanism to address complexity induced within an active distribution network. In this way, it can provide a pragmatic solution for demand flexibility scheduling and dispatch. Moreover, the agile methodology can empower customers by ensuring the fair distribution of DR benefits and by representing their value propositions in the wholesale market.
Section 2.4 contributes a description of transactive control mechanism (TCM) and the implicit research required towards sustainability and resilience in the imminent
transactive energy. In particular, TCM has been chosen for the agile methodology because it can promote attributes of flexibility, bidding, connectivity, empowerment, for balancing global and local system objectives through adaptive control topology. Section 2.5explores a hybrid architecture of multi-agents andSection 2.5.1exploits it by describing micro-models of applied agents. Section 2.5.1 identifies the significance of energy bidding in the agile methodology and presents rules for the composite bidding instead of the simple ones. Furthermore, the principle of distributed learning is presented inSection 2.7.
2.4 t r a n s a c t i v e c o n t r o l m e c h a n i s m
Forfia et al. in [45] define Transactive Energy as an emerging concept for explicit DR while considering grid reliability constraints. They further state that the Transactive Energy fundamentally applies to a coordination mechanism of value propositions/bids captured in transactions between cognitive entities (called agents) within a community (called multi-agent environment). In this way, an individual agent is not exposed to complex rules, and communication burden is relatively lower [46]. Hence, a Transactive Control Mechanism (TCM) can be conceived as a market-based mechanism for DR where agents are organized to offer bids for control.
Research in the TCM is strongly interdisciplinary, involving the state-of-the-art in power engineering, computer science, economics and control engineering.Table 2.3presents the selected research that assesses the transition of the present trends to the transactive energy. It has been inferred fromTable 2.3that an agent-based transactive control mechanism is a solution to improve market responsiveness as well as to cope complexity induced within active distribution networks. This section, therefore, extends the description of the TCM by describing its attributes, which are also necessary for agility to emerge.
2.4.1 Attributes of Transactive Control
In light of the literature reviewed inTable 2.3, the consistent attributes drove the evolution of the TCM are as follows:
f l e x i b i l i t y In TCM, customers always have the right to override DR through altering a control space of a physical appliance presented by an appliance agent. The control space, which is a set of variables, is specific to primary process parameters of an appliance and is required to generate an optimal bid.
b i d d i n g Every agent in the TCM should communicate with the control parameters via a single value signal (bid)1
. The bid is not necessary to reflect actual energy price used for revenue and billing purposes. Moreover, the bid may consist of control parameters and/or costs to be weighed relatively in the form of a local normalized currency. Above all, bids should be communicated explicitly throughout the system. Thus a price signal or an incentive in response to the bids agents is transacted as a final control decision.
1 Hereinafter, term “bid" will be used instead of a value signal that reflects actual value propositions by an
2.4 transactive control mechanism 15 o r g a n i z a t i o n Decentralize Centralize Hierar chical Hybrid Fr equency based Short et al. [ 47 ]; Mallada et al. [ 48 ] Pour mousa vi et al. [ 49 ] ; T ro v ato et al. [ 50 ] V rettos et al. [ 51 ] Dir ect load W ang et al. [ 52 ]; Geor ges et al. Geor ges et al.; and Xue et al. [ 54 ] For ecast based Essakiappan et al. [ 55 ]; Hanif et al. [ 56 ] Lampr opoulos [ 57 ]; Long et al. [ 58 ] Price based Gy amfi et al. [ 59 ] W ang et al. [ 60 ]; Y oon et al. [ 61 ] Incentiv e based Elmenr eich et al. [ 62 ] Farahani et al. [ 63 ]; Babar et al. [ 64 ] Babar [ 33 ] Market based Negne vitsky et al. [ 19 ]; T orbaghan et al. [ 23 ] Peppink et al. [ 65 ] Haque et al. [ 66 ] co nt ro ls T ransactiv e Y ue et al. [ 67 ] Kok et al. [ 46 ]; De Craemer [ 34 ]; Babar et al. [ 36 ]; Babar et al. [ 68 ] T able 2. 3: Descriptions of contr ol mechanisms in vie w of system or ganizations v erses DR contr ol strategies.
e m p o w e r m e n t 2 Analyzing the real-time demand and its control empowers an agent to make its bids/value propositions. In general, the TCM is given empowerment to set its objectives, thus having full control over its agents. Therefore, as a system, the TCM can dispatch demand continuously, not merely for a few critical stressed periods of the year.
c o n n e c t i v i t y TCM allows multiple communication media, protocols, and vendors to coexist and compete. Moreover, the hierarchical and transactive control reduce overall communication bandwidth due to a reduction in an amount of information flowing throughout the hierarchy.
m u lt i p l e o b j e c t i v e s The TCM has been influenced by the operational objectives and DR benefits, which value most for Smart Grids. Among the most important are:
• Demand Flexibility It measures the responsiveness of an agent to the changing environment. By applying demand elasticity in bidding mechanism, more efficient agent behaviors can be achieved in the short term.
• Demand Flexibility Schedule It improves system efficiency by deferring, pre-empting or both the use of dispatchable appliances to achieve objectives (like cost minimization). Moreover, it facilitates the consumption of DERs. For instance, electric vehicles (EVs) or storage can be charged when energy is discounted or generated nearby. Chapter 4 provides case studies of an application of demand flexibility scheduling using the TCM in the context of Agile DR.
• Demand Flexibility Dispatch It mitigates operational constraints by dissuading the consumption if served power threatens to damage or shorten the asset’s life time. In the TCM, price or incentive signal should be permitted to rise when the local constraint is violated. In this way, the higher price signal persuaded agents to either use less power from or provide distributed generation to the grid. Similarly,Chapter 5provides an application of demand flexibility dispatch for congestion management in the context of Agile DR by using the TCM. a g e n t t e c h n o l o g y Currently, agent technologies are emerging in demand response
programs because they offer flexibility, distributed, openness, responsiveness, redundancy, and scalability. There are numerous works in the field of DR which use the Multi-Agent System (MAS) for demand flexibility acquisition, scheduling, and dispatch [34, 46, 65–67]. Kok in [69] demonstrates the principle of the-state-of-the-art hierarchical MAS based TCM. Therein, a bid from each agent flows upstream to an aggregator, correspondingly price signal flows downstream to the agents. The given organization of agents is specified to the formulation of bids at the last downstream agents, and DR only occurs at the most upstream agent. However, the intermediate agents do not have any intelligence. They provide just an abstraction of data by aggregating the bids of the associated downstream agents.
2 Hereinafter, term “real-time" refers to the time intervals and future horizon that must be selected to
accommodate and influence dispatch decisions. The preferred time interval should be short enough to enable innovative bidding services while supported by existing communication technologies.
2.5 three layered hybrid architecture 17
Until now, the most of the findings from the TCM facilitates the introduction of the agile methodology and provides its theoretical background. However, the agile methodology and its development need to understand as a whole.
2.5 t h r e e l ay e r e d h y b r i d a r c h i t e c t u r e
Many relevant studies, as mentioned in tab:tcm, utilize hierarchical MAS in different DR strategies, where each agent possesses a strategy to achieve single or multiple objectives of the TCM continuously. In this way, If agents and their objectives, strategies, and interactions are design carefully within the TCM, an Agile DR paradigm can be leveraged to achieve three prime goals of agility (i.e., the improved responsiveness, the process adaptability and the customer enabling). The improved responsiveness in agility is initially a concept which realizes the current primary process parameter through the advanced technologies. The customer enabling in agility is a way of integrating customer uncertainty in the design. Instead of minimizing uncertainty with detailed requirements and specifications, it enables the customer by taking feedback or preferences on a predictable timeline. However, the process adaptability in agility is a method that combines an iterative and incremental learning in the strategic processes with a focus on the customer’s preferences.
Therefore, to achieve a combination of the prime goals by the most important objectives of the TCM, agents in the MAS are organized in three strategic layers. Moreover, within the layer the MAS allows the agents to share the strategic objectives and knowledge about the environment explicitly. This set of multiple agents is referred to as Three Layered Hybrid Architecture and found the most suitable organization for the agile methodology. The strategic layers can be understood by the concept, as presented inFigure 1.2. Each agent in a layer showed agility but limited to a particular strategic process, technology, and customer retention. As shown inFigure 2.2, for an agent in a layer, the customer enabling is based on its knowledge about the surrounding environment. For instance, in case of an end-consumer, the control space by the appliance agent provides knowledge to help in strategic decisions. However, the strategic processes are algorithms that use the available expertise and make a decision based on a particular objective and a focused, agile goal. Depends on the available technology at hand, an agent has to fix its combination of goal and objective. Referring to theFigure 2.2, the strategy of an agent relates to its strategic process based on a particular objective. However, actions are taken by an agent depends on its knowledge and technology.
On the other hand, the architecture is hybrid due to two main reasons, as shown in Figure 2.3. Firstly, agents across different strategic layers possess different goals and objectives. Moreover, each agent performs its tasks nearly autonomously and communicate via the hierarchical transactive control. Secondly, the architecture can readily partition tasks or workloads among agents at the same layer in a distributed fashion. Thus, the interaction between the agents is mapped to explicit mechanisms for peer-to-peer communications.
OBJECTIVE
Agent Interaction Strategic Layer Complete Environment
Figure 2.2: Multi-Agent System.
2.5.1 Micro-Modeling
Within Smart Grid, the benefits of interoperability are widely recognized. Within interoperability in urban energy management, the most important recent developments are arguably the IEEE 2030 standard [70], NIST Framework and Roadmap for Smart Grid Interoperability Standards [71] and Energy Flexibility Platform and Interface (EF-Pi) [72]. These advancements provide essential groundwork towards interoperability in energy systems, by facilitating a common understanding and aiding the development of interoperable systems. Critically, the guide highlights the importance of standard models in promoting data exchange and the integration of energy systems, and in enabling system security and performance. Further, the vital role of standard models is highlighted in ensuring a shared meaning of data, hence making the data more useful, as well as providing inference and rule-based functionality.
Alongside the requirement for the three-layered hybrid architecture, the use of a knowledge-based ontology mitigates the effort required for the standardization of the Agile DR strategies for the development of the agile methodology. Models which elucidate these functional modeling of knowledge management are at this moment referred to as Micro-Models.
Micro-modeling within agent-oriented programming is instantiated with specific agents and their interactions to create a comprehensive knowledge base. Figure 2.4 shows a UML Class Diagram as a conceptual model that illustrates the agents of the agile methodology for the behavior of software implementation using agent-oriented programming. It includes the structural agent operation, its run-time instance execution,
2.5 three layered hybrid architecture 19
Application Agent Coordinating Agent Component Agent Hierarchical Cordination Distributed Cordination
Demand
Elasticity ScheduleDemand DispatchDemand
Customer Enabling Process Adaptibility Improved Responsiveness LA YERS STRATEGIES GOALS Application Layer Cordination Layer Access Layer
Figure 2.3: Three-Layered Hybrid Architecture for the agile methodology.
the communication agent message, and traces, which organize related performances or interactions. The timed scenarios of the agents’ interactions with peer agents are captured by a UML sequence diagram as shown in Figure 2.5. It displays two UML sequence diagrams generated for day-ahead and real-time execution of the methodology. Moreover, in the sequence diagram, messages between agents are only an implicit representations of the starting requests and the ending responses. The boxes represent the functional responsibilities of the agents, which will be discussed in details in the subsequent chapters.
Although the dissertation only discusses the micro-modeling of the proposed agents, further agents can also be micro-modeled with the paradigm. The following parts of the thesis build the micro-models of the agents because they are specific to the assigned tasks to achieve agility goals and objectives. Hence, they will be developed and explained in this thesis. Briefly, Chapter 3will discuss the models of an appliance agent and its instances as well as Logger and Elasticity agents. On the other hand, an aggregator agent (i.e., an application agent) to solve the demand flexibility scheduling problem will be presented in detail inChapter 4and micro-models Allocation and Grouping agents.Chapter 5will extensively model network, percept, dispatch and host agents. Therein, the Network agent is an application agent that interacts with the aggregator agent to solve demand flexibility dispatch problem.
