1 / 16

Adaptive Stochastic Control for the Smart grid

Adaptive Stochastic Control for the Smart grid. Qinghua Shen Smart grid meeting. Outline. Introduction to the control of smart grid adaptive stochastic control, smart grid Adaptive Stochastic Control basis of stochastic system, policy search and approximation, convergence

oriel
Download Presentation

Adaptive Stochastic Control for the Smart grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Adaptive Stochastic Control for the Smart grid Qinghua Shen Smart grid meeting

  2. Outline • Introduction to the control of smart grid adaptive stochastic control, smart grid • Adaptive Stochastic Control basis of stochastic system, policy search and approximation, convergence • Example: distributed generation despatch with storage ADP for resource allocation, value function approximations • Challenges

  3. Introduction to the control of smart grid • Control of the smart grid • goal for control: instantly, corrective and dynamically • Self-healing: auto repair or removal of potentially faulty equipment • Flexible: rapid and safe interconnection of distributed generation and storage • Predictive: statistics, machine learning, predictive models • interactive: appropriate information is provided transparently in near real time • Optimal : operators and customers efficiently and economically • Secure : cyber- and physical-security

  4. Introduction to the control of smart grid • Major Components

  5. Outline • Introduction to the control of smart grid adaptive stochastic control, smart grid • Adaptive Stochastic Control basis of stochastic system, policy search and approximation, convergence • Example: distributed generation despatch with storage ADP for resource allocation, value function approximations • Challenges

  6. Adaptive Stochastic Control • Stochastic system • State variables • physical state: energy amount, status of a generator • information state: current and historical demand, price and weather • belief state: probability distributions • The decisions • whether or charge/discharge, use backup • The exogenous Information • all the dimensions of uncertainty • The Transition Function • given the state, decisions and exogenous information, determines next state • The Objective Function • metrics that governs how we make those decisions and evaluate the performance of policies of the controller designs

  7. Adaptive Stochastic Control • Policies • maps the information in state S to a decision x. • , which is the state variables, capturing energy resources Rt, exogenous information pt, and belief state Kt. • The problem • is known variously as the value of the a policy or the cost to go function. • can be a cost function if we minimize, or contribution function if we maximize. • Cost include generating electricity, purchasing fuel, losses due to energy conversion, cost of repair, and penalties for curtailing loads • policy for what • includes whether to charge/discharge, when to run a distributed generator, how much energy draw from grid for every customer in every networks and the utility

  8. Adaptive Stochastic Control • Design a robust policy: four classes • Myopic Policies • minimize next-period cost without decisions for future( special structure good) • Look-ahead Policies • Optimize over some time horizon using a forecast of the possible variability of exogenous events such as weather. Forecast can either be deterministic forecasts or stochastic forecasts • Policy Function Approximations • Functions return an action given a state, without solving any form of optimization, including: rule-based lookup table; Parameterized rules(threshold hold); statistical functions • Policy based on Value Function Approximations • Optimal policy obtained from HJB equation, to avoid curses of dimensionality • a) Approximate to eliminate the expectation; b) replace the value function with a computationally tractable approximation; c) solve the resulting deterministic maximization problem using a commercial solver

  9. Adaptive Stochastic Control • ADP and the Post–Decision State • Value function approximation • when structure of a policy is not obvious, estimates the value of being in a state • When x is a vector, solve the maximization problem is problematic(expectation hard to compute exactly)---refer to stochastic search • Post-Decision state • Post decision state determined through current state and action

  10. Adaptive Stochastic Control • Design policy • look up tables • Parametric models • With this strategy, we face the challenge of first identifying the basis functions, and then tuning the parameters • Nonparametric models • handle high-dimensional, asymptotically unbiased • • Kernel regression; • • Support Vector regression; • • Neural networks; • • Dirichlet process mixtures.

  11. Adaptive Stochastic Control • Policy search • Direct policy search • Depend on Monte Carlo sampling ----stochastic search • Methods: sequential kriging • Using the knowledge gradient • Applied when the policy structure is apparent • Bellman residual minimization for value function approximations • This is the most widely used strategy for optimizing policies, and encompasses a variety of algorithmic approaches that include approximate value iteration (including temporal difference learning) and approximate policy iteration

  12. ASC for distributed generation despatch • Approximate Dynamic Programming for resource allocation • Resource allocation • how much energy to store in a battery, whether a diesel generator should be turned on, and whether a mobile storage device (and/or generator) should be moved to a congested location. • A general model • Rta is the number of resources with attribute vector a • xtad is the number of resrouces we act on with a decision of type d. • a decision d can be (-1,0,1) to discharge, hold, or recharge a battergy • (0,1) to turn a distributed generator off or on

  13. Adaptive Stochastic Control • Value Function Approximations for resource allocation • Approximate the value function • resource allocation utility function: concavity property • Approximate value function by the post decision resource vector Separable piece-wise linear function • Estimate piecewise linear concave functions by iteratively stepping forward through time and updating value functions

  14. Adaptive Stochastic Control • Experimental work • evaluate the results • Resource determine the quality of the resulting policy is a major challenge • fit the value functions for a deterministic problem, and compare the resulting solution to the optimal solution for the deterministic problem, obtained by using a commercial solver • limited by the size

  15. Challenges • Convergence • Only some structure can be proofed to be convergence with approximation • Concavity is an important category • For smart grid • Beneficial to both utility and end users– enough incentive • The track of key performance metrics

More Related