- 123 Views
- Uploaded on
- Presentation posted in: General

Adaptive Stochastic Control for the Smart grid

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Adaptive Stochastic Control for the Smart grid

Qinghua Shen

Smart grid meeting

- Introduction to the control of smart grid
adaptive stochastic control, smart grid

- Adaptive Stochastic Control
basis of stochastic system, policy search and approximation, convergence

- Example: distributed generation despatch with storage
ADP for resource allocation, value function approximations

- Challenges

- Control of the smart grid

- goal for control: instantly, corrective and dynamically
- Self-healing: auto repair or removal of potentially faulty equipment
- Flexible: rapid and safe interconnection of distributed generation and storage
- Predictive: statistics, machine learning, predictive models
- interactive: appropriate information is provided transparently in near real time
- Optimal : operators and customers efficiently and economically
- Secure : cyber- and physical-security

- Major Components

- Introduction to the control of smart grid
adaptive stochastic control, smart grid

- Adaptive Stochastic Control
basis of stochastic system, policy search and approximation, convergence

- Example: distributed generation despatch with storage
ADP for resource allocation, value function approximations

- Challenges

- Stochastic system

- State variables
- physical state: energy amount, status of a generator
- information state: current and historical demand, price and weather
- belief state: probability distributions

- The decisions
- whether or charge/discharge, use backup

- all the dimensions of uncertainty

- given the state, decisions and exogenous information, determines next state

- metrics that governs how we make those decisions and evaluate the performance of policies of the controller designs

- Policies

- maps the information in state S to a decision x.
- , which is the state variables, capturing energy resources Rt, exogenous information pt, and belief state Kt.
- The problem
- is known variously as the value of the a policy or the cost to go function.
- can be a cost function if we minimize, or contribution function if we maximize.
- Cost include generating electricity, purchasing fuel, losses due to energy conversion, cost of repair, and penalties for curtailing loads

- policy for what
- includes whether to charge/discharge, when to run a distributed generator, how much energy draw from grid for every customer in every networks and the utility

- Design a robust policy: four classes

- Myopic Policies
- minimize next-period cost without decisions for future( special structure good)
- Look-ahead Policies
- Optimize over some time horizon using a forecast of the possible variability of exogenous events such as weather. Forecast can either be deterministic forecasts or stochastic forecasts

- Policy Function Approximations
- Functions return an action given a state, without solving any form of optimization, including: rule-based lookup table; Parameterized rules(threshold hold); statistical functions

- Policy based on Value Function Approximations
- Optimal policy obtained from HJB equation, to avoid curses of dimensionality
- a) Approximate to eliminate the expectation; b) replace the value function with a computationally tractable approximation; c) solve the resulting deterministic maximization problem using a commercial solver

- ADP and the Post–Decision State

- Value function approximation
- when structure of a policy is not obvious, estimates the value of being in a state
- When x is a vector, solve the maximization problem is problematic(expectation hard to compute exactly)---refer to stochastic search

- Post-Decision state
- Post decision state determined through current state and action

- when structure of a policy is not obvious, estimates the value of being in a state

- Design policy

- look up tables
- Parametric models
- With this strategy, we face the challenge of first identifying the basis functions, and then tuning the parameters
- Nonparametric models
- handle high-dimensional, asymptotically unbiased
- • Kernel regression;
- • Support Vector regression;
- • Neural networks;
- • Dirichlet process mixtures.

- Policy search

- Direct policy search
- Depend on Monte Carlo sampling ----stochastic search
- Methods: sequential kriging
- Using the knowledge gradient
- Applied when the policy structure is apparent

- Bellman residual minimization for value function approximations
- This is the most widely used strategy for optimizing policies, and encompasses a variety of algorithmic approaches that include approximate value iteration (including temporal difference learning) and approximate policy iteration

- Approximate Dynamic Programming for resource allocation

- Resource allocation
- how much energy to store in a battery, whether a diesel generator should be turned on, and whether a mobile storage device (and/or generator) should be moved to a congested location.

- A general model
- Rta is the number of resources with attribute vector a
- xtad is the number of resrouces we act on with a decision of type d.
- a decision d can be (-1,0,1) to discharge, hold, or recharge a battergy
- (0,1) to turn a distributed generator off or on

- Value Function Approximations for resource allocation

- Approximate the value function
- resource allocation utility function: concavity property
- Approximate value function by the post decision resource vector Separable piece-wise linear function
- Estimate piecewise linear concave functions by iteratively stepping forward through time and updating value functions

- Experimental work

- evaluate the results
- Resource determine the quality of the resulting policy is a major challenge
- fit the value functions for a deterministic problem, and compare the resulting solution to the optimal solution for the deterministic problem, obtained by using a commercial solver
- limited by the size

- Convergence
- Only some structure can be proofed to be convergence with approximation
- Concavity is an important category

- For smart grid
- Beneficial to both utility and end users– enough incentive
- The track of key performance metrics