Research and Experimental Design

1. Research and Experimental Design Edward O. Garton, Jon S. Horne, Jocelyn L. Aycrigg, and John T. Ratti 2012 Department of Fish and Wildlife Sciences, University of Idaho, Moscow, Idaho Chapter 1 In Silvy, Nova (ed.). Techniques For Wildlife Investigations and Management. Seventh edition, Vol. I Wildlife Research Methods. The Wildlife Society, Bethesda, MD.

2. Crisis! • There is a crisis in wildlife, fisheries, ecology and conservation biology. • Our practice of scientific method is broken. • Our rigor in designing research has been lost in a rush to investigate. • We have failed to lay the proper foundation for experiments, models and samples. • Even worse, we have failed to confront our understanding (theories) with data to improve and correct them.

3. Twelve-Step Program for Recovering Naturaholics to Bring Back Rigor • 1. Identify problem and set objectives. • 2. Conduct literature review and contact experts. • 3. Collect preliminary observations and apply exploratory data analysis to them. • 4. Formulate a theory (conceptual model or research hypothesis). • 5. Formulate multiple predictions from conceptual model as testable hypotheses or alternate models. • 6. Design tests/experiments/models to test the theory.

4. Twelve-Step Program (cont.) • 7. Do a pilot study (techniques, variances, costs). • 8. Revise proposal with a statistician based on results of pilot study and reviews by experts. • 9. Do experiment, collect observations and/or build model. • 10. Confront theories with data (analyze it). • 11. Evaluate, interpret and draw conclusions. • 12. Use results and publish them for all to use and build on.

5. Scientific Method (Conceptual)

6. Overview: Formal Sequence of 18 Steps Step Example What are the influences of environmental factors, such as wildfire and winter severity, on the carrying capacity of elk winter range? Excellent earlier work by Houston 1982, Merrill & Boyce 1991, … Determine temporal and spatial differences in food… • Identify the research problem. • Conduct literature review of relevant topics. • Identify broad and basic research objectives. See additional steps and remainder of example in Box 1.1 of book chapter.

7. Initial Steps • Problem Identification • Applied research • Basic research • Literature Review • Professional Society Publications: • TWS, AFS, ESA, SCB, SAF, SRM, others • Google Scholar, Web of Science, etc. • Use Google, Yahoo and other general search engines cautiously and critically

8. Initial Steps: Defining Populations (Biological, Political, and Research) • Delimiting populations • Defines potential applications (inference space) • Required for sampling • Equally important for experiments (esp. field experiments) and models

9. Hierarchically Structured Biological Units: • Individual • Deme • Population • Metapopulation • Subspecies • Species

11. Deme • “A group of individuals where breeding is random” (Emlen 1984). • “A panmictic population” (Ehrlich and Holm 1963) • Identification of demes and other groupings of individuals should be based on demography, movement, genetics and geography of individuals and habitats.

12. Deme - Definition • The smallest grouping of individuals approximating random breeding, within constraints of breeding system, where it is reasonable to estimate birth, death, immigration and emigration rates.

13. Deme • Genetics: Random breeding within constraints of social system. • Demography: Smallest grouping where its feasible to estimate birth, death, immigration and emigration rates. • Movement: Restricted to home ranges in key seasons. • Geography: Continuous distribution of individuals within one patch of habitat or a closely spaced set of habitat patches.

14. Population • A collection of demes or individuals at one point in time, typically the breeding season, with strong connections demographically (very high correlations in vital rates), geographically (close proximity), genetically (Manel et al. 2005) and through frequent dispersal.

15. Population • Movement: High rates of dispersal between adjacent demes. • Genetics: Very closely related genetically. • Geography: A collection of closely-spaced occupied patches of habitat without great expanses of non-habitat intervening between them. • Demography: Very high correlations between demes and small groups of individuals.

16. Metapopulation • A collection of populations sufficiently close together that dispersing individuals from source populations occasionally colonize empty habitat resulting from local population extinction (Levins 1969). • Populations within a single metapopulation may show low or high correlations in demographic rates, but the low rates of dispersal are still high enough to maintain substantial genetic similarity. • Numerous types of metapopulations have been described from source-sink to non-equilibrium to Levins’ classic (Harrison and Taylor 1997).

17. Metapopulation • Movement: Probability of dispersal between populations is low but colonization occurs. • Demography: Possible low correlations in rates produces high independence whereas demographics of “source” populations may drive demography of “sink” populations. • Genetics: Genetic differentiation occurs between populations but not enough to become subspecies. • Geography: Substantial areas of non-habitat may separate populations and patches of suitable but unoccupied habitat.

18. Biological, Political, and Research Populations

19. Statistical/Research Population = Sampling Universe = Sampling Frame • Entire set of units (i.e., elements, potential observations) that exist and from which we want to make inference. • Even though we want to make statements (i.e., estimates) for animal or plant populations we often recast this population spatially: • i.e., statistical population = all the spatial units occupied by our biological population

20. Statistical Population • GIS a key tool for handling sampling frame because we often lay it out spatially. • We use sampling frame to define statistical population to which we apply our estimates • We also draw our samples randomly from the sampling frame. • But note that randomly doesn’t mean stupidly – we can draw them cleverly while we draw them randomly.

21. Sampling or Experimental Units • Non-overlapping collections of elements of statistical population that cover entire statistical population. • Note that these sampling units may be individuals, plots, spatial polygons, or collections of elements.

22. Initial Steps • Define Population(s) • Preliminary Data Collection • Get our feet wet and try out ideas • Exploratory Data Analysis • Search literature for others’ ideas • Put on thinking cap • Theory, Models, Predictions and Hypotheses

23. Conceptual Model of Waterfowl Population Dynamics

24. Components of Theory (after Pickett et al. 2007) • Domain: Scope in space, time and phenomena addressed by a theory. • Assumptions: Conditions needed to build theory. • Concepts: Labeled regularities in phenomena. • Definitions: Conventions and descriptions necessary for the theory to work with clarity

25. Components of Theory • Confirmed generalizations: Condensations and abstractions from a body of facts that have been tested or systematically observed. • Laws or principles: Conditional statements of relationship or causation, statements of identity, or statements of process that hold within a domain.

26. Components of Theory • Models: Conceptual constructs that represent or simplify the structure and interactions in the material world. • Scientific models can project consequences of ideas • Statistical models draw inferences and discriminate between competing ideas based on limited observations

27. Components of Theory • Framework: Nested causal or logical structure of a theory. • Translation: Procedures and concepts needed to move from the abstractions of a theory to the specifics of applications or test or vice versa. • Hypotheses: Testable statements derived from or representing various components of theory.

28. Study Designs for Investigation Process

29. Experiments Strengths and weaknesses of different types of experiments (modified from Diamond 1986). __________________________________________________________ Experiment Type __________________________________________________________ Laboratory Field Natural _____________________________ Control of independent variablesa Highest Medium Low Ease of inference High Medium Low Potential scale (time and space) Lowest Medium Highest Scope (range of manipulations) Lowest Medium High Realism Low High Highest Generality Low Medium High __________________________________________________________ aActive regulation and/or site matching. See Table 1.1 in book chapter.

30. Experimental Design • Focus of statistics programs, especially in agricultural regions • Dependent experimental units • Fixed, random, mixed, and nested effects • Controls • Replication • Determining required sample size

31. Checklist for Experimental Design • 1. What is hypothesis to be tested? • 2. What is response or dependent variable(s) and how should it be measured? • 3. What is independent or treatment variable(s) and what levels of variable(s) should we test? • 4. To which population do we want to make inferences?

32. Checklist for Experimental Design • 5. What will be our experimental unit? • 6. Which experimental design is best? • 7. How large should the sample size be? • 8. Have you consulted a statistician and received peer review on your design?

33. Modeling • Powerful alternative for: • solutions to pressing problems • selecting best of alternatives • effects from multiple simultaneous causes • evaluating population viability • Goal: build simplest scientific model withability to produce useable predictions • Statistical models used for estimation, hypothesis testing, and comparison

34. Scientific Modeling Strategies DescriptionSimpleComplex Quantification Conceptual (verbal)------------Quantitative Theoretical General-----------------Complex Simulation Relationships Linear------------------------------Non-linear Variability Deterministic----------------------Stochastic Time Scale Time-specific------------------------Dynamic Mathematical Difference ------------------------Differential Formulation Equations Equations Factors Single------------------------------Multifactor Spatial Single Site---------------------------Multi-site No. of Species Single Species------------------Multi-species

35. Statistical Modeling Strategies DescriptionSimpleComplex Sampling Simple random----------Stratified, clustered, or multi-stage Hypothesis testing Fixed or -----------------Mixed fixed and random random effects effects Independence of Complete ----------------Dependence between observations independence observations in space, time, or both Errors Single term---------------Separate process and observation errors

36. Steps to Build a Scientific Model • Problem definition • System identification • Select model type • Mathematical formulation • Parameter estimation • Model validation • Model experimentation

37. Basic Concepts of Survey Sampling • Survey sampling is a magic sword; • but it’s a two-edged sword: • Sharp edge slices through all kinds of problems producing unbiased estimates even if some assumptions about population are wrong • Dull edge requires applying some type of random sampling

38. Designing a Survey • 1. What is objective of survey? • 2. What is best technique? • 3. To which population will we make inferences? • 4. What will we sample (sample unit)? • 5. What is the size of population we are sampling (N)? • 6. Which sample design is best? • 7. How large should sample be?

39. Sampling Designs and Types of Sampling Units or Experimental Units

40. Key Terms and Concepts • Precision: deviation of repeated samples from each other. • Bias: deviation of repeated samples from true value. • Accuracy: deviation of an individual sample from true value.

41. Bias, Precision, and Accuracy

42. Confronting Theories with Data • Hypothesis testing (classic frequentist) • Effect size and interval estimation • Information theoretic model selection • Regression and General Linear Models (GLM - Fisherian) • Bayesian Approaches • Validating parametric and simulation models

43. Confronting Theories with Data

44. Finishing • Speculation and new hypotheses • Publication will • Correct errors • Insure sound conclusions • Facilitate defending controversial practices • Help personnel grow in skills • Make a permanent contribution to wildlife knowledge

45. Common Problems to Avoid • Procedural inconsistency • Non-uniform treatments • Pseudoreplication • Insufficient sample size

46. Adaptive Management: Connecting Research and Management • Specify management goals formally • Use predictive models to: • Summarize existing knowledge of factors and processes producing desired responses • Forecast results of alternative management actions • Select best management action to implement, treating it as an experiment to evaluate • Monitor what happens to: • Ensure goals are met • Correct relationships and processes assumed in management

47. Adaptive Management: Connecting Research and Management • Alternatives: • Structured Decision Making • Scenario Planning • Various forms of business decision-making

48. Summary • Carefully designed wildlife research will improve reliability of knowledge base of wildlife management. • Research biologists must rigorously apply scientific method and make use of powerful techniques in survey sampling, experimental design and information theory. • We must move from observational studies to experimental studies, replicated across space and time, that provide a more reliable basis for interpretation and conclusions. • Modeling is an effective tool to predict consequences of management choices, especially when based on carefully designed field studies, long-term monitoring, and management experiments designed to increase understanding.

49. Summary • Wildlife biologists have tremendous responsibility associated with management of animal species experiencing: • increasing environmental-degradation problems • loss of habitat • declining populations • We must face these problems armed with knowledge from quality scientific investigations.