資源管理與動態分析 第 2 講 模式的驗証與評估

1. 資源管理與動態分析 第2講 模式的驗証與評估 Ref:Tanya M. Shenk, Alan B. Franklin, Modeling in natural resource management, Island press, Washington, 2001 簡報人: 陳炳宏

2. 簡報內容 • 1 Introduction • 2 Purposes of Models • 3 Kinds of Models • 4 Examples of Models • 5 Verification and Sensitivity Analysis • 6 Validation & Evaluation Methods

3. 1 Introduction Model every paper published in journals invokes models applying modeling methodology to the ecological sciences Texts such as Grant et al. (1997) and Starfield and Bleloch (1991) Modeling: modern methods accessible in the classroom, and computer software

4. 1 Introduction Models abstractions of reality model's purpose drive from Jamestown to Zap, North Dakota, need an abstraction of the state such as highways but not average precipitation a road map, which is one form of a model attract ducks to within shooting distance a decoy

5. 1 Introduction predict mallard recruitment based on information about nest success and wetland conditions a mathematical model The appropriate form and content of a model depend on the model purpose the validation issue

6. 2 Purposes of Models models applications include summarizing information defining a problem organizing thinking Communicating Most purposes fall into one of three categories: Explanation Prediction decision making

7. 2 Purposes of Models Models categories differ according to output values finding input values based on the other two pieces : Input > MODEL > Output

8. 2 Purposes of Models Models for explanatory purposes seek to explain a system's behavior understand the phenomena involved to identify the system given input and output conditions Identifying and understanding the causal mechanisms of the system sometimes called understanding, learning, causal, or rational models.

9. 2 Purposes of Models Models for prediction used to forecast the outcome of the system seek values of the output based on the input conditions Engineers and managers find these models most useful. Models for decision making used to prescribe the right input to achieve desired outputs to optimize some product of a system Also called prescriptive or control models

10. 3 Kinds of Models Models can be classified in Verbal Diagrammatic Physical formal

11. 3 Kinds of Models Models can be split along dichotomies deterministic versus stochastic physical versus abstract dynamic versus static discrete versus continuous (in time homogeneous versus heterogeneous Model resolution vary in scope and detail finer grained and intensive to coarse-grained In General: mechanistic or descriptive model

12. 3 Kinds of Models Mechanistic models incorporate components and interactions also called functional, causal, explanatory, or rational models most useful for explanatory purposes Descriptive models no claim to being accurate mimics intended to simulate relationships between input and output values also called correlational or statistical models

13. 3 Kinds of Models "empirical model" Wrong leading on implying that descriptive models have a basis in observations that mechanistic models lack Descriptive models invoked for predictive or decision-making Hybrid models Mechanistic models very useful predictions When input variables are outside the range of values used to develop and fit a predictive model

14. 4 Examples of Models The scores of elementary school children on a test of arithmetic skills to the students‘ body weight (Figure 7.la Another model (Figure 7.\b) relates test scores to age A more mechanistic model relates test performance to the student's amount of training (Figure 7.1c

15. 4 Examples of Models

16. 4 Examples of Models

17. 4 Examples of Models

18. 4 Examples of Models explaining the causal mechanisms for input variables outside the observed range We might expect performance improve with receiving more training.

19. 4 Examples of Models energy and mass could then develop a statistical model good predictive ability The true model is E = mC2 formula represents a mechanistic model Einstein obtained without benefit of regression

20. 4 Examples of Models

21. 4 Examples of Models In ecology we rarely have such clean and elegant models Species-area models fairly close (Figure 7.3) More often, we have statistical or descriptive models

22. 4 Examples of Models Figure 7.3

23. 4 Examples of Models • The mallard renesting model • Figure 7.4 shows for renesting in mallards • If a female mallard is unsuccessful in a nesting attempt, she may try again and, following a second failure, perhaps again • the probability that a female is successful in hatching a clutch during a breeding season • we call hen success, H • the probability that an individual nesting attempt is successful (nest success, P

24. 4 Examples of Models Figure 7.4 Equation: H=0.03+1.54*P

25. 4 Examples of Models regression model fits the data well (R^ = 0.966) but not really explain what is going on Outside the range, the model not predict well If nest success were 1, hen success would be 1.57  impossible

26. 4 Examples of Models model (Cowardin and Johnson 1979): make some reasonable assumptions about rates of renesting H=Pe(i-P)^2 as does the regression model, but it is a mechanistic model complex models involving flowcharts use computer simulation flowcharts would look Hke Figure 7.5

28. 5 Verification and Sensitivity Analysis Verification & validation, Figure 7.6 Verification to determine if the model behaves as it was intended or computer code is correct Sensitivity analysis used to evaluate models involves systematically and comprehensively testing to see how changes in the model's parameters suggests levels of confidence

29. 5 Verification and Sensitivity Analysis

30. 6 Validation & Evaluation Methods Validation Marcot et al. (1983) identified 23 criteria Precision Generahty Accuracy Robustness resolution a better alternative to the term validation: evaluation

31. 6 Validation & Evaluation Methods Subjective Assessment not compare model and true values the use of expert opinion asking knowledgeable persons to assess the model prone to personal bias should be avoided

32. 6 Validation & Evaluation Methods Visualization Techniques comparing time series of real observations and model results plotting actual values against predicted values graphing differences between actual and predicted values still subjective may be misleading

33. 6 Validation & Evaluation Methods Measures of Deviation applicable when actual data (y) and modeled data (y) can be paired by time deviation based on the difference between y and y A useful measure is the root mean square error: RMSE

34. 6 Validation & Evaluation Methods Statistical TesU involves the use of F tests correlation coefficients other statistical tools to compare actual and model results

35. 提問討論