Quantitative Research Method (Research Design)

Quantitative Research Method (Research Design) By TemtimAssefa October, 2013

Overview of Quantitative Research • Uses deductive method of knowledge acquisition • Intends to falsifies an existing theory • Tries to generate generalizable knowledge • Accepts objectivity of knowledge • Uses a standard measurement instruments • The researcher is independent in the process of knowledge construction • Intends with prediction and controlling

Research Process Problem Data Interpretation Conceptual FW, Hypothesis Research Design Data Collection & Analysis Develop Instrument

Step 1: Problems/Questions A question/problem occurs or is posed to the researcher for which that researcher has no and can find no answer(s). Example – usability The software development industry has invested in the development of a variety of Usability Inspection Methods (UIMs) to address Web usability issues (Matera et al., 2006). UIMs are procedures in which inspectors examine usability-related aspects of a user interface (Rocha and Baranauska, 2003). However, in (Rivero et al., 2013), we identified that there is a lack of UIMs that are able to find usability problems in early stages of the development process.

Step 2: Literature, Theory, Logic Available literature/theory is reviewed to determine if there is already a solution to the problem. Existing solutions do not always explain new conditions. The existing solution might require some revision or even be discarded. Develops a hypothesis as a tentative solution to the problem Stone (1956) defines hypothesis as a tentative explanation about as the relationship between two or more variables

Empirical to theoretical mapping

Conceptual Framework - User Involvement in system development Participation Conflict Conflict resolution Influence

Types of Quantitative Research Design • A research Design is nothing more than a plan for conducting a research in such a way as to answer a research questions. • A number of questions arise to select the research design – who, what shall be the focus of the study? What attributes of individuals, organizations, systems, etc, In what contexts observations will be made (organizations, laboratories, etc), what methods employed to analyze the data?

Cont’d … • Researchers are constrained with people, time, money, etc to select the best the research design • A prime consideration is designing cost effective research • Selecting a research design Quantitative research method design includes • Survey /field study • field experiment, • laboratory experiment, • simulation, and • case study

Decide form of data collection • Questionnaires • Interviews • Observation

Questioners • Most frequently used measurement instrument in organization studies • Includes sets of questions to which the subject responds • It is used to measure: • Attitude • Opinions • Demographic characteristics of the subjects

Questionnaires design • Questionnaires make use of lists and rating scales • Behaviors and attitudes are complex and cannot be easily evaluated and quantified • Check list is a list of behaviors, characteristics or other entities that a researcher is investigating • Either the researcher or participants simply check(s) items from the list • What are features of user friendly software • Graphical interface • Clear navigation direction • Immediate feedback • Other specify ____________

Questionnaire … • A rating scale is more useful when a behavior, attitude, or other phenomena of interest needs to be evaluated on a continuum scale • It is designed with the following scales • “inadequate” to “excellent”, • “never” to “always” or • “strongly disapprove” to “strongly approve”

Guideline for Questionnaire • Keep it short • What do I do with the information? • Is it absolutely essential to have this information to solve part of the research problem • Use simple, clear, unambiguous language • Check for unwarranted assumptions implicit in your questions • How many cigarette do you smoke each day? • Good to add a choice • 25 ___ 25-16 ____ 15-5 ___ <5 ____ None ____ • Word your questions in ways that don’t give clues about preferred or more desirable responses What strategies have you used to try to quit smoking? • Leads him to list strategies he did not try

Guide line … • Check consistency – that leads to give contradictory answer for two questions • Determine in advance how you will code the response • Keep the respondents task simple • Provide clear instructions • Make the questionnaire attractive and professional looking • Conduct a pilot test • Give for half a dozen friends to see they have difficulty understanding any items • Scrutinize the almost final product carefully to make sure it address your needs

Advantage of Questionnaire • Easily to collect data from large number of people, including those who live thousands of miles away • Can be administered by unskilled data collectors • Presents a uniform stimulus to all subjects • Can be collected online or through mail • Researcher does not have contact with the respondents • The anonymity of the respondents helps respondent to provide more truthful information than they would be in personal interviews, especially on sensitive and controversial issue

Draw back of questionnaire • Majority of people may not return, if it is mailed the return rate is not more than 50% • Returned responses may not be a true representative of the originally selected samples • Cannot be used with illiterate or persons who cannot read • Most questioners are inflexible, respondents answer by choosing only from the given lists

Interviews Interviews are meetings in which the interviewers directs questions at the interviewee and records the obtained responses It is direct face to face interactions Can be undertaken through telephone Help to captured more detailed information than questioners Allow to include the respondents views

Design of Interview • Can be structured and unstructured • Unstructured interview is designed with open ended questions • The researchers asks and the responds reply to the questions • The same questions may not be asked for different respondents • Examples • Can you list down the benefit of the new software for your organization? • What are the main drawbacks of the new software? • What do you suggest to improve software to solve the current problems?

Structured interviews • The researcher asks standard set of questions and nothing more • In semi structured interview, the researcher may follow the standard questions with one or more individually tailored questions to get clarification or probe a persons reasoning • In structured interview, you can have checklist of responses • Example • What are the benefits of LAN in the organization • Facilitates communications among employees • Improves information access • Improves relationship with customers • Other specify • What kind of the support the management provides to the success of the Intranet system

Advantages of Interview Interview is formal and emotionally neutral Help to develop hypothesis at early stage of the research than testing hypothesis It has a distinct advantage of establishing rapport with potential participants and gain cooperation Has higher response rate Flexible to include additional views and opinions Has better validity, the researcher can further ask probing questions on some biased response

Disadvantage It is generally expensive than other data collection methods The researcher is the main interviewer otherwise training of interviewer is very expensive and time taking The interviewer presents the questions in different manners that lead to different responses – affects the validity and reliability of the data Characteristics of the respondents (age, sex, ethnicity, etc) may affect the value of measured variables Respondents may not free on some controversial issues

Data Analysis and Interpretation • Does the observed data supports the previously postulated research hypothesis/relationships between variables • There are different data analysis (statistical ) methods for data analysis • Commonly mentioned as descriptive and inferential statistical methods • Descriptive methods include: • Frequency, mean, • median, • correlation, • Charts and graphs, etc

Data analysis … • Inferential methods include: • Parametric test: t-test, ANOVA, linear regression, • Non-parametric test: Chaisquare test, Man Whitney test

Benefits • Theoretical propositions can be tested in an objective fashion • Enable generalizations • Easy to use (although may be an illusion) • Established and accepted research method across multiple fields • Aided by statistical tools • Immensely helped by advancement computing • Strong in external validity if sampling errors are minimized.

Field Experiment

Field Experiment • A field experiment is an experimental research method which is performed outside the laboratory in the natural settings • It follows all steps of scientific process • selection and determination of a problem, • selection of participants and measuring instruments, • selection of a research plan, • execution of the plan, • analysis of data and • formation of conclusions

Features of FE • It has three unique features • the research takes place in natural setting • the experimenter manipulates one or more independent variables while exerting much control over confounding variables; and • the effect of the manipulations on one or more dependent variables is systematically observed.

There three type of designs FE Design

Controlling Nuisance in Field Experiment • Solomon Four Groups in True Experiment Experimental group 1: R O X O Control group 1: R O O Experimental group 2: R X O Control group 2: R O • Removing statistically the effect of suspected confounding variables

Example – Interface Design and Computer use System use is low among employees. One reason is poor interface design. You divide the employees into two groups (Group 1 & 2) Redesign the Interface and let Group 1 work on the new system - Treatment Group Let Group 2 work on the old Interface– Controlled Group After six or some months after Interface design, measure, the computer use. If computer use is increased for Group 1, you can conclude that Interface design and computer use has association

Data collection and Analysis • The following methods are used to collect data • Questionnaire • Structured interview • Computer log files • Data analysis • Descriptive statistical methods • Inferential statistical methods

Advantages of FE • Conducted in natural settings and has higher experimental validity • Help to clearly identify antecedents of observed effects in casual relationships • Used for the development of theory as well as for the solution of applied problems; • The logic of FE can be applied in the analysis of many naturally occurring changes • Help for testing of broad hypotheses dealing with complex social process in lifelike situations.

Disadvantage • The methods used to control confounding variables in FE are not sufficient • Manipulation of variables cause legal and ethical problems • Difficult to control dynamically changing environment during the course of the experiment • Difficult to precisely measure dependant variables in the field settings • Expensive to conduct compared to lab experiment • Needs highly skilled person to design and conduct

Role of FE in computer Science • It has many applications in IS • To develop theory • To test hypothesis • To evaluate IS tools and techniques

Practical Examples • Fukada et al (n.d.) used field experiment to study the effect of Road Facility Management Support System on asset management of the public infrastructure. • Mayur et al (2000) made study to compare and suggest the best training method for companies between instructor based training and computer based training methods. • Chen et al (2007) made a field experiment study to identify effects of different types of social information on contributions to an online communities.

Review Questions Explain the different quantitative research design methods What is the strength and weakness of each method What are the different survey research methods What is the main data analysis methods what problem can you address with quantitative research methods What are the procedures in quantitative Research How do you use Quantitative research in computer science?

Quantitative Research Method (Research Design)