1 / 31

Select Question

Overview of Research Process. Ethics. Develop Hypothesis. Research Design. Review Research. Communicate Results. Data Analyses. Select Question. Measurement. Current Focus. Research Designs. Max Precision. Max Context. Runkey & McGrath typology. Max Generality. Data Analyses.

nash
Download Presentation

Select Question

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of Research Process Ethics Develop Hypothesis Research Design Review Research Communicate Results Data Analyses Select Question Measurement

  2. Current Focus Research Designs Max Precision Max Context Runkey & McGrath typology Max Generality Data Analyses Scaling Measurement Reliability & Validity Qualitative

  3. Runkel & McGrath, 1972 Contrived Settings Obtrusive Operations Maximum Precision Experimental Simulations Lab Experiments Field Experiments Setting Independent Natural Settings Field Studies Sample Surveys Maximum Context Computer Simulations Formal Theory Unobtrusive Operations Maximum Generality Behavior not measured Universal Behavior Systems Particular Behavior Systems

  4. How to do field research?

  5. What is field research? • Examples • Field Studies • Cross sectional • Field Experiments • E.g., Longitudinal, prog evaluation • Similarity and differences from • Other methods of data collection • Large-scale (Sample) Surveys • Methods statistical analyses • Correlational

  6. Runkel & McGrath, 1972 Contrived Settings Obtrusive Operations Maximum Precision Experimental Simulations Lab Experiments Field Experiments Setting Independent Natural Settings Field Studies Sample Surveys Maximum Context Computer Simulations Formal Theory Unobtrusive Operations Maximum Generality Behavior not measured Universal Behavior Systems Particular Behavior Systems

  7. Why do field research?: General reasons Describe Predict Explain

  8. Why do field research? Organization-specific reasons • Type of organizational change & development • E.g., self & peer evaluation of oral presentation (Radhakrishnan & Yang, 2006) • Two-way (symbolic) communication channel between employees & organization via content and conduct • e.g., UT Employee Survey • Cox, T. Jr (2001). Creating the Multicultural Organization: A Strategy for Capturing the Power of Diversity San Francisco, CA: Jossey Bass

  9. After deciding why you are doing field research, decide how you will collect data • Types of Data Collection Methods • Numerical vs. Non-numerical • Oral/Written vs. Observational • Behavioral vs. non-behavioral • Each of the above types of data can be collected via all or some of the following • Questionnaires/Surveys • Observation (Archival) • Interviews

  10. Methods of data collection • Bias in any one method is overcome if you use multiple methods • Cf choosing research designs • Some methods are better suited for measuring certain kinds of concepts • E.g., willingness & ability should determine use of self report • Stereotype research • Amount of resources used by method • Researchers resources • Participants’ resources

  11. Time & resources restrict you to certain methods of data collection • Questionnaires • E.g., Field study, cross-sectional data • Archival data • E.g., Field studies, Sample (large scale) Surveys • If using, justify measures w/logic & research • e.g., ESL indicators • Qualitative (non-numerical) data will take too long for collection & analyses

  12. Instructor-Generated Exampleof a Questionnaire • Hypothesis based on Rode et al., 2005, AOMLE • Additional control variable • Renner, M. & Mackin R. (2000). A life stress instrument for classroom use in M. Ware & D. Johnson (Eds.) M. Handbook of demosntrations and activities in the teaching of psychology: Vol 1 Lawrence Erlbaum: Marwah, NJ.

  13. Before designing your questionnaire identify • Research hypothesis • Predictor, criterion & explanatory variables • Pre-existing measures of predictor & criterion variable • Bonus if you have measure of explanatory variable

  14. Why identify pre-existing measures for your questionnaire? • Examples of pre-existing measures • Found in books on Reserve at CIRHR library • Psycinfo database: • Search: Measures OR Questionnaires AND your topic keyword • Why use pre-existing measures • Improves statistical reliability of your study • Improves validity of your study • Disadvantages of pre-existing measures • E.g., UT study

  15. How pre-existing measures improve validity • Validity • Content based on definition of concept • Content can be based on qualitative data generated by potential participants • E.g., critical incidents for ethnic harassment (EH) measure (Schneider, Hitlan, & Radhakrishnan 2000) but see Swim et al EH measure • Not all constructs need participant-generated data • e.g., answers to an exam

  16. How pre-existing measures improve reliability • Reliability • If measure is tested on samples similar to your sample, then you can be confident in the measure • Schneider et al., 2000 • Can reasonably expect hypothesis to be supported if concepts are reliably measured

  17. Pre-existing measures used in the instructor’s example • Satisfaction measures • cited in Rode et al., 2005 • Performance Measure • Cited in Rode et al, 2005 • Control Variables • Citizenship replaced by primary language question which is more appropriate • Not feasible to collect IQ measure in context • Stress measure • Described in Renner & Mackin, 2000 Instructor slightly modified stem based on previous research (Schneider et al., 2000)

  18. After deciding on measures, structure questionnaire • Content of Study information Sheet & Consent form • See methodology assignment guidelines • Logic of ordering • Assess criterion variable first in cross-sectional study • Attractiveness via Visual Layout • Headings, Font size, White Space

  19. More issues to consider when structuring questionnaire • Number of control variables & length of survey • Shortening pre-existing measures is tempting but might damage reliability and validity. • Assessing sensitive variables • E.g., Class demonstration survey; UT survey • Ease of data analyses • Numbering sections & items • Number of Open-ended questions

  20. Issues the Instructor faced when designing the examplar questionnaire • Sensitive Variables • Dropping additional demographic variable due to sample size • What if the hypothesis is not supported • Restricted range on the GPA variable • Arguments to use stress as a control variable vs. an antecedent

  21. While or After designing questionnaire develop sampling plan • Sampling plan depends whether you want maximum precision, maximum context or maximum generality • E.g., maximum generality then need random, large, representative sample

  22. Runkel & McGrath, 1972 Contrived Settings Obtrusive Operations Maximum Precision Experimental Simulations Lab Experiments Field Experiments Setting Independent Natural Settings Field Studies Sample Surveys Maximum Context Computer Simulations Formal Theory Unobtrusive Operations Maximum Generality Behavior not measured Universal Behavior Systems Particular Behavior Systems

  23. Some terms in the area of sampling • Population: • Group you are interested in obtaining data from and studying. • Sample: • Representative number of respondents from the population that you sample. • Actual sample: • The actual number of participants from your sample that complete and return your survey

  24. Types of Sampling You Can Hope vs. Actually do

  25. Random Sampling • Every person in the population has exactly the same probability of being included in the sample to avoid bias. • Sample is representative of the larger population. • Representativeness can be checked by comparing the characteristics of a sample to those of the population • e.g., gender, age, tenure

  26. One Possible Modification of Random Sampling • Stratification sampling: • Population divided into groups called strata. • Random selection from within groups. • Ensures representation on some critical factor in the sample (e.g., gender, job category).

  27. A Second Possible Modificationof Random Sampling • Cluster sampling: • Participants chosen as members of a group rather than as individuals. • Randomly select work teams, organizations, factories, plans, facilities, etc.

  28. Convenience Sampling(AKA what you will end up doing for this course) • Selection of participants based on easy availability or accessibility. • Snowball or chain sampling – people who know people.

  29. How to get a good sample size • Provide incentives before or after. • Indicate support from stakeholders. • Convincing reason to complete it. • Promise of feedback. • Reminders. • Personalize correspondence. • Return envelope with postage / web-survey

  30. What you learned today • Is your study a field study (or field expt) or a sample survey? • Will you administer the questionnaire yourself or collect archival data? • For both data collection methods you need to use data collected with, or collect data with pre-existing valid & reliable measures • How to find reliable & valid measures • Why use them

  31. What you learned today • How to design a good questionnaire • What sampling plan you can hope to use • How to get a large enough sample with the sampling plan you will use

More Related