1 / 13

Reformulated Neural Network ( ReNN ) A New Alternative for Data-driven Modelling

Reformulated Neural Network ( ReNN ) A New Alternative for Data-driven Modelling in Hydrology and Water Resources Engineering. Saman Razavi 1 , Bryan Tolson 1 , Donald Burn 1 , and Frank Seglenieks 2 1 Department of Civil and Environmental Engineering,

nell
Download Presentation

Reformulated Neural Network ( ReNN ) A New Alternative for Data-driven Modelling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reformulated Neural Network (ReNN) A New Alternative for Data-driven Modelling in Hydrology and Water Resources Engineering Saman Razavi1, Bryan Tolson1, Donald Burn1, and Frank Seglenieks2 1 Department of Civil and Environmental Engineering, University of Waterloo, Waterloo, Ontario, Canada 2 Environment Canada, Burlington, Ontario, Canada

  2. Outline of the Presentation Introduction to Reformulated Neural Network Application 1 – Metamodelling Application 2 – Rainfall-Runoff Modelling A new Measure of Regularization Summary 2

  3. Reformulated Neural Network Multilayer Perceptron(Traditional Neural Network) ReNN is: • Essentially a single-hidden-layer neural network • Defined on a new set of variables based on the network’s internal geometry • Main Features: • ReNN is more efficient in training • ReNN variables are interpretable • ReNN is more predictable in generalization 3

  4. Reformulated Neural Network ReNN variables in 1-input problems Hwi.Hoi a sigmoidal unit Slope 1 si = Hwi. Iwi,1 Height Hwi Hwi.Hoi Location di = -Hbi/Iwi,1 x1 4

  5. Reformulated Neural Network ReNN variables in 2-input problems Directional Slopes Slope Height Hw1Ho1 New Variables: Height Location & Slope Angle x2 x1 Angle Location 5 Directional Slope 1 Directional Slope 2

  6. ReNN in n-input problems is non-trivial but for details seeRazavi and Tolson (2011) Details include: Generalized geometry & revised neural network formulation with respect to the new variables Derive the partial derivatives of the network error function with respect to the new variables for back-propagation training algorithms Razavi, S., and Tolson, B. A. (2011). "A new formulation for feedforward neural networks." IEEE Transactions on Neural Networks, 22(10), 1588-1598, DOI: 1510.1109/TNN.2011.2163169. • Example Applications … 6

  7. ReNN Efficiency in Training - Case Study 1 • Example Application in Metamodelling Neural networks are frequently used to model (emulate) computationally expensive models (e.g., in optimization, model calibration, real-time/ operational settings) Network Training efficiency is very important. SWAT2000 Hydrologic Model Cannonsville Reservoir Watershed, NY Test Function trained with Standard Back-propagation Alg. trained with Standard Back-propagation Alg. Averaged over 50 trials Averaged over 50 trials Saving 7

  8. Interpretation of ReNN Variables – Case Study 2 • Example Application in Rainfall-Runoff Modelling • (monthly) Precipitation Gauge 1(t) Precipitation Gauge 2(t) ReNN 6-5-1 Runoff (t) Precipitation Gauge 1 Precipitation Gauge 3(t) Precipitation Gauge 4(t) Precipitation Gauge 2 Average Precipitation (t-1) Walton Runoff Gauge Precipitation Gauge 3 Average Temperature (t) Precipitation Gauge 4 Input-output data are scaled to [-1 +1] Cannonsville Reservoir Watershed New York (area = 1200 km2) 8

  9. Interpretation of ReNN Variables – Case Study 2 Overall Slopes Directional Slopes Locations Output Bias Heights Precipitation Gauge 1 1 Precipitation Gauge 2 0.8 Walton Runoff Gauge -16 16 Precipitation Gauge 3 Precipitation Gauge 4 -1 Cannonsville Reservoir Watershed New York (area = 1200 km2) 9

  10. Interpretation of ReNN Variables – Case Study 2 Overall Slopes Directional Slopes Locations Output Bias Heights Precipitation Gauge 1 Output Bias 8.56 1 Precipitation Gauge 2 Walton Runoff Gauge -16 16 Precipitation Gauge 3 2.65 Precipitation Gauge 4 -1 8.65 Cannonsville Reservoir Watershed New York (area = 1200 km2) 10

  11. ReNN Regularization Measure regconventional= This measure directly quantifies the smoothness of the network response. Among two networks with the same accuracy on the training data, the one with smoother response (more regularized) is expected to have better generalizability Conventional measure of regularization: regnew= This measure is applicable to both ReNN and traditional neural networks Performance function with regularization: Performance Function = *mse + * regnew w (1 - w) 11

  12. Summary Reformulated Neural Network (ReNN) is an equivalent reformulation of multilayer perceptron (MLP) neural networks with the following benefits: ReNN is trained faster, ReNN has an interpretable Internal Geometry – e.g. useful for Sensitivity Analysis, and ReNN has a direct measure of regularization (smoothness). ReNN can turn into traditional neural network. For full information on ReNN formulation and derivations, please refer to: Razavi, S., and Tolson, B. A. (2011). "A new formulation for feedforward neural networks." IEEE Transactions on Neural Networks, 22(10), 1588-1598, DOI: 1510.1109/TNN.2011.2163169. 12

  13. Thank You For full information on ReNN formulation and derivations, please refer to: Razavi, S., and Tolson, B. A. (2011). "A new formulation for feedforward neural networks." IEEE Transactions on Neural Networks, 22(10), 1588-1598, DOI: 1510.1109/TNN.2011.2163169.

More Related