SCIENTIFIC research

1. SCIENTIFIC research Dr. Yan Liu Department of Biomedical, Industrial and Human Factors Engineering Wright State University

2. Examples of Human Factors Research Questions • How can different types of flight simulator visual scene detail affect detection of altitude change? • How do team and work characteristics impact team functioning? • Can a virtual character placed in an educational environment motivate active learning of students? • What factors are important in designing group support systems? • What aspects of individual differences significantly affect users’ interaction with a complex computer program? • What are differences between interaction in real world and that in virtual reality?

3. Scientific Research • Why Human Factors Engineers Study Scientific Research Methods • Research is important when developing and assessing the effectiveness of products designed to achieve certain goals • Problems with Intuition • Numerous cognitive and motivational biases affect our perceptions and lead to erroneous conclusions about cause and effect • Illusory correlation occurs we focus on two events that stand out and occur together or when we are highly motivated to believe in the causal relationship • A scientific approach requires much more evidence before conclusions can be drawn

4. Scientific Research (Cont’d) • Scientific Skepticism • Ideas must be evaluated on the basis of careful logic and results from scientific investigations • Empiricism • The fundamental characteristic of the scientific method • Knowledge is based on observations • Data are collected that form the basis of conclusions • The scientific method embodies a number of rules for collecting and evaluating the data

5. Elements of Scientific Inquiry(Goodstein’s “Evolved Theory of Science” (2000)) • Observations are Accurately Reported to Others • Others can replicate the methods used and obtain the same results • Fabricating data is inherently unethical and dealt with by strong sanctions • Search for Observations for Verification of Ideas • Develop hypotheses • Argue that existing data support the hypotheses • Conduct research that can increase the confidence in the correctness of the hypotheses • Open Exchange and Competition among Ideas • Good scientific ideas are testable • Peer Review of Research • Makes sure research with major flaws will not become part of the scientific literature

6. Goals of Scientific Research • Description of Behavior • Researchers begin with careful observation to describe events • Researchers are often interested in describing the ways in which events are systematically related to one another • Prediction of Behavior • If two events are systematically related to one another, it is possible to make predictions • When an operator is under high workload, his performance may suffer • Determining the Causes of Behavior • Predicting behaviors ≠ Identifying causes • Aptitude test scores do not cause college grades. The aptitude test is an indicator of other factors that are the actual causes • To know how to change a behavior, we need to know the causes of the behavior

7. Goals of Scientific Research (Cont’d) • Explanation of Behavior • Understand why the behavior occurs • Further research may be necessary to shed light on possible explanation of the causation relationship

8. Necessary Conditions of Causation • Temporal Precedence • There is a temporal order of events in which the cause precedes the effect • Covariation of the Cause and Effect • When the cause is present, the effect occurs; when the cause is not present, the effect does not occur • Elimination of Alternative Explanations • Only the causal variable could be responsible for the observed effect

9. Basic and Applied Research • Basic Research • Fundamental questions about the nature of behavior • Address theoretical issues concerning basic phenomena • cognition, emotion, motivation, learning, personality development, etc. • Applied Research • Conducted to address issues in which there are practical problems and potential solutions • Design • Evaluation • Comparing Basic and Applied Research • Both are important • Progress in science depends on a synergy between the two • Applied research is often guided by theories and findings of basic research • Findings obtained in applied settings often require modification of existing theories and spur more basic research

10. When research participants were asked to recall common and rare words presented in separate lists, they recalled more of the common words. However, if the same words were presented in mixed lists, the participants remembered more rare words. When both are presented together, the study strategy is apparently to focus on the rare words. (Watkins et al. 2000) A knowledge-based, interactive visualization system was developed, with the aim to enable physicians and medical support personnel to draw conclusions from heterogeneous time-oriented clinical data. Experimental results suggested that the proposed system could facilitate semantically-based browsing. (Cheng et al. 2005)

11. Hypotheses • Hypothesis • A tentative idea or question that is waiting for evidence to support or refute it • Informal hypothesis • Simple question derived from researcher’s conjecture or observations • Do males and females differ in their attitudes toward online shopping? • A procedure is developed to collect data to answer the question • Conduct a survey of male and female online shoppers • Formal hypothesis • Two or more variables are related to one another • Crowding results in reduced performance on cognitive tasks • Attending to more features of something to be learned will result in greater memory • Formulated on the basis of past research findings or theoretical considerations • A study is designed to test the hypothesis • Place research participants in either a crowded or uncrowded room to work on a series of tasks and then observe their performance

12. Predictions • Prediction • A statement concerning what will occur in a particular research investigation • Participants in the uncrowded condition are expected to perform better on the tasks than those in the crowded condition • If the prediction is confirmed by the results of the study, the hypothesis is supported (NOT proven); otherwise, the hypothesis is rejected • A variety of methods may be used to test a hypothesis; the more times a hypothesis is supported by studies, the more likely the hypothesis is correct

13. Who We Study: A Note on Terminology • Participants (APA Recommended) • Individuals who participate in research projects • Subjects (Seen in Some Articles) • Equivalent to participants • Respondents • Individuals who take part in survey research • Informants • People who help researchers understand the dynamics of particular cultural and organizational settings • Originated in anthropological and social research, but is sometimes used by psychologists as well

14. Sources of Ideas • Common Sense • Things that we all “believe” to be true • “A picture worth a thousand words” • Testing a commonsense can be valuable because such notions do not always turn out to be correct • Pictures can aid memory under certain circumstances, but they sometimes detract learning • Conducting research to test commonsense ideas often forces us to go beyond a commonsense theory of behavior • Observation of the World around Us

15. Sources of Ideas (Cont’d) • Past Research • Becoming familiar with a body of research on a topic (literature review) is perhaps the best way to generate ideas for new research • Refine and expand our knowledge • Identify questions that can be addressed in subsequent research • Investigate inconsistencies in research results • Study alternative explanations for the results • Apply the findings in a different setting (e.g. different population, different tasks, etc.) • Practical Problems • Problems that have immediate applications • The development of virtual reality technology was motivated by people’s desire to overcome the limitations of standard computer monitors

16. Your Term Project General Flowchart of Research Process

17. ACM Transactions on Computer-Human Interaction; Applied Cognitive Psychology; Applied Ergonomics; Behaviour & Information Technology; Computers in Human Behavior; Communications of the ACM; Ergonomics; Ergonomics in Design; Human Computer Interaction; Human Factors; Human Performance; IEEE Transactions on Systems, Man, and Cybernetics; Interactions; International Journal of Aviation Psychology; International Journal of Human-Computer Interaction; International Journal of Human-Computer Studies; International Journal of Industrial Ergonomics; Journal of Applied Psychology; Memory and Cognition; Perception and Psychophysics; Psychological Review Journals Related to Human Factors

18. APA Style • Website: http://apastyle.apa.org/ • Set of specific formatting conventions sanctioned by the American Psychological Association (APA) • Widely used by researchers in psychology, human factors engineering, and behavioral and social sciences • An sample article and a paper of “APA Style Essentials” are posted on the course website

19. Organization of a Research Article in APA Style • Title Page • Abstract • Main Body • Introduction (not labeled) • Method • Results • Discussions • References • Appendix • Footnotes • Tables • Figure Captions • Figures

20. Title Page • Short Title • The first two or three words of the full title • Next to the page number in the header for the purpose of identifying which paper the pages belong to in case they accidentally become separated • Running Head • First line of the title page • Entirely capitalized and should be flushed left • An abbreviated title (no more than 50 characters, including spaces) • Should the article be published in a journal, the running head would appear at the top center of every other page of the article in the journal • Title • Brief and descriptive of your research • No more than 15 words • Clearly states what the research is about • Avoid jargon or vernacular • Author Byline • Institutional Affiliation

21. Abstract • A Brief Summary of the Article (100 – 150 words) • Include the most important aspects of the study within the provided word-limit • Purposes • Helps readers decide whether to read the article • Provides readers with a framework for understanding the article if they decide to read it • Content • Statement of the problem and primary hypotheses • Information on the characteristics of the participants and a brief summary of the procedure • Patterns of findings for major variables • Direction of differences without relying on numerical values of the variables • Implications of the study

22. Introduction • Statement of Problem under Study • Broader context of the problem • Significance of the study • Literature Review • Shows how your research builds on prior knowledge by presenting and evaluating the previous related research • Describe only the research and theoretical issues that are clearly related to your study • State explicitly how the previous work is logically related to your research problem • Rationale and Hypotheses of the Study • What variables you are studying and what results you expect • The links between the research hypotheses, prior research, and the current research design are shown by explaining why the hypotheses are expected

23. Method • Purpose • Provides detailed information about how your study was conducted • The description is sufficient if enough detail is presented for the readers to replicate the essential elements of the study • Overview (for complex experimental design and procedures) • An overview of the method to prepare the readers for the information that follows • Participants • Number of participants • Nature of the participants (gender, age, ethnicity and other relevant characteristics) • Special characters (e.g. freshman engineering students, IT professionals) • How participants were recruited and what the incentives for participation were (e.g. cash, course credits, coupons) • Describe with sufficient detail so that it is clear what population the participants represent; this is needed for replicabilility of the study and understanding its generalizability/transferability

24. Method(Cont’d) • Apparatus (for special equipment) • The brand name and model number of the equipment • Procedure • Describes exactly how the study was conducted • What instructions were given to the participants • How the independent variables were manipulated • How the dependent variables were measured • How extraneous variables were controlled • Other experiment design issues (e.g. counterbalancing, randomization, etc.) • The method of debriefing • For a non-experimental method, provide details on exactly how you conducted the study and the measurement techniques used • Other subsections • If they are needed for clear presentation of the method

25. Results • Purpose • Describe data analyses and their results • Complications of the Analysis • Description of any complications (e.g. missing data, outliers, etc.) and how you handled them • Statistical Techniques • Understand the techniques applied and the statistics you are reporting • Try to use the simplest, appropriate technique that meets the underlying assumptions • If inferential statistics are used, you should determine the power a priori based on your anticipated distribution, design, and definition of practical significance. This information should come from a similar previous study or a pilot study, rather than the data that you collected for statistical analysis. If data fail to show statistical significance, then the power analysis can be used to show that the finding is not due to a low statistical power

26. Results (Cont’d) • Provide the summary details about the findings • Use carefully planned tables and graphs

27. Discussions • Discuss and interpret research findings from various aspects • Whether the results support the hypotheses • If they do, give all possible explanations for the results and discuss why one explanation is superior to another • If they do not, suggest possible reasons (e.g. wrong methodology, wrong hypotheses, or both) • Compare the findings with those of previous studies • Suggest possible practical applications of the research • Recognize the limitations of the study • Suggest future research on the topic

28. Other Sections • References • Start on a new page • There should be a one-to-one match between the references cited in the report and the list of references • See the article “APA Style Essentials” for APA reference style • Appendix • Start on a new page • Seldom used in manuscripts submitted for publication • Necessary materials that would be distracting in the main body of the article • A complex mathematical proof • A questionnaire or survey instrument • Footnotes • Start on a new page • Should be avoided unless they are absolutely necessary

29. Other Sections (Cont’d) • Tables and Figures • Each should be on a separate page • Tables and their captions are shown together • Figures are placed after the tables and a separate page containing the figure captions is provided before the figures