Robert Froese, Ph.D., R.P.F. School of Forest Resources and Environmental Science

1. Validating Wykoff's Model, Take 2:Equivalence tests and spatial analysis in a design-unbiased analytical framework Robert Froese, Ph.D., R.P.F. School of Forest Resources and Environmental Science Michigan Technological University, Houghton MI 49931 http://www.teresco.org/pics/signs/20010627/forest.jpg

2. This presentation has six parts • The model and the objectives • The region, the data and the approach • How well does Wykoff’s model predict? • Does Wykoff’s model pass a validation test? • How does accuracy relate to model and data structures • What does it mean for model users and future revisions Introduction Methods Performance Equivalence Trends Relevance

3. This presentation has six parts • The model and the objectives • The region, the data and the approach • How well does Wykoff’s model predict? • Does Wykoff’s model pass a validation test? • How does accuracy relate to model and data structures • What does it mean for model users and future revisions Introduction Methods Performance Equivalence Trends Relevance

4. This presentation has six parts • The model and the objectives • The region, the data and the approach • How well does Wykoff’s model predict? • Does Wykoff’s model pass a validation test? • How does accuracy relate to model and data structures • What does it mean for model users and future revisions Introduction Methods Performance Equivalence Trends Relevance

5. This presentation has six parts • The model and the objectives • The region, the data and the approach • How well does Wykoff’s model predict? • Does Wykoff’s model pass a validation test? • How does accuracy relate to model and data structures • What does it mean for model users and future revisions Introduction Methods Performance Equivalence Trends Relevance

6. This presentation has six parts • The model and the objectives • The region, the data and the approach • How well does Wykoff’s model predict? • Does Wykoff’s model pass a validation test? • How does accuracy relate to model and data structures? • What does it mean for model users and future revisions Introduction Methods Performance Equivalence Trends Relevance

7. This presentation has six parts • The model and the objectives • The region, the data and the approach • How well does Wykoff’s model predict? • Does Wykoff’s model pass a validation test? • How does accuracy relate to model and data structures • What does this mean for model users and for future revisions? Introduction Methods Performance Equivalence Trends Relevance

8. Wykoff’s model predicts basal area increment but is used to project diameter DDS = DBH2t+10 - DBH2t BAG = (π/4)·(DBH2t - DBH2t-10) DG = (DBH2 + DDS)0.5 - DBH ln(DDS) = f( SIZE +SITE +COMP) Introduction Methods Performance Equivalence Trends Relevance

9. Wykoff’s model is a multiple linear regression on the logarithmic scale • bi – coefficients estimated by ordinary least squares, of which: • b0 depends on habitat type and nearest National Forest • b2 depends on nearest National Forest • b12 depends on habitat type

10. This validation is focused on two notions • Caswell (1976) introduces two ideas: • does a model user care if the internal structures are truthful, as long as the model makes accurate predictions? • does the scientist care if the model makes accurate predictions, as long as the model is useful for testing hypotheses about the underlying system? • Robinson and Froese (2004) question how statistical tests are used for model validation • The usual null hypothesis is of no difference, or that a model is valid, which seems unscientific • Arbitrarily small differences are detectable • A failure to reject may simply imply low power

11. This study had four objectivesand two perspectives The objectives were: • to estimate model bias by species across the range of application; • to demonstrate a specific validation of Wykoff’s model for diameter increment prediction through a test of equivalence; • to identify significant trends between bias and predictor variables, and; • to evaluate spatial trends in bias across the geographic area to which Prognosis is usually applied. Two perspectives were taken regarding Wykoff’s model: • as a diameter increment model, and; • as it contributes to predictions of per hectare volume increment, which is more intuitive or of more interest to many forest managers.

12. National Forestsand geography ofthe Inland Empire Introduction Methods Performance Equivalence Trends Relevance

13. Inland Empire forests change predictably at various geographic scales 3 4 1 2 1, 2, 4: http://www.tarleton.edu/~range/Woodlands%20and%20Forest/Northern%20Rocky%20Mountain%20Forests/NorthernRockyMountainForests.htm 3: http://www.flintridgefoundation.org/conservation/feature_pic1l.jpg

14. Data came from Forest Inventory and Analysis Subject is prediction error Impute volume increment by backdating Vt - Vt-10 =∆V Correct for log transform bias + The focus in this study was on geographically extensive individual tree field data

15. Equivalence tests flip theburden of proof onto the model • Select a metric of model performance • Nominate an interval of equivalence  • Say 10% of • Construct two one-sided confidence intervals of size  • If completely contained within the interval, reject the null hypothesis of dissimilarity From Robinson and Froese (2004)

16. Most FIA plots were variable probability samples and may imply a design bias Introduction Methods Performance Equivalence Trends Relevance

17. Design unbiased results showed a modest over prediction by Wykoff’s model Extrapolated to the study area, over prediction could be: ~2,400 ha·plot-1 • 2,632 plots • 0.5 m3·ha-1·dec-1 = ~3,158,400 m3·dec-1

18. Equivalence tests are constructed within a regression framework Introduction Methods Performance Equivalence Trends Relevance

19. Equivalence tests for diameter increment generally fail to validate the model

20. For stand level volume increment, equivalence tests frequently validate the model

21. Prediction error is weakly related to most predictor variables Introduction Methods Performance Equivalence Trends Relevance p ≤ 0.01, r2 ≤ 0.1 in all cases

22. Diameter prediction error shows a spatial trend irregularly correlated with elevation

23. Spatial trends in volume prediction error largely mirror those for diameter

24. In some locations bias appears meaningfully different on and off of National Forest lands Forests that are equivalent have an obvious matrix of public and private land across elevation and geography

25. Wykoff’s model for prediction • Equivalence tests provide an objective methodology for assessing model validity • There is added subjectivity in the selection of I • For diameter, a large I would be necessary to validate Wykoff’s model • For most species I = 25% would have to be used • largely because of bias not two-one-sided CI • For volume, the model is largely validated • But trends show bias is close to zero for average conditions • Overprediction of > 3 mil m3•dec-1 is not insubstantial • Species results differ, and may imply invalid stand dynamics Introduction Methods Performance Equivalence Trends Relevance

26. Wykoff’s model as a theory • Wykoff’s model is surprisingly robust • These tests involve substantial extrapolation in time and space • Model improvements should focus on the way climate is represented • LOC as a proxy for regional climate • EL as a global parabolic function • Interactions with other predictors, like DBH2 • Static proxies or process variables? • Other issues remain • Small trees

27. Summary Wykoff’s model modestly over predicts diameter increment, but the effect on volume is smaller ∆D 14% ∆V 2% Equivalence tests fail to validate the model for diameter increment, but less often for volume As a theory, the model is surprisingly robust The way climate is represented in the model needs to be addressed