Educational Research

1. EducationalResearch Chapter 8

2. Tools of Research • Scales and instruments – measure complex characteristics such as intelligence and achievement • Scales and instruments may be: • (1) Selected or (2) Developed by the researcher • If selected: • Look at previous research into the problem and see what scales and instruments were used • Mental Measurement Yearbook • Tests in Print • Educational Testing Service – available on WWW

3. Tools of Research Types of scales and instruments: • Achievement Tests – standardized or researcher-made • Performance Assessments • Aptitude Test – individual and group • Measures of Personality • Attitude Scales – summated rating scales and bipolar adjective scales • Rating Scales • Direct Observation

4. Tools of Research cont Achievement Tests – measure mastery and proficiency in different areas of knowledge (e.g., California Achievement Test in reading, language, arithmetic) Standardized Tests – can compare the results of studies that use the same tests • Comparative norms – representative and relevant sample • Validity • Reliability • Directions for administering and scoring

5. Achievement Tests cont • Researcher-made achievement tests • It is better to construct your own instrument that use one that is standardized but inappropriate for your purpose • If you construct your own instrument, need to establish its validity and reliability by administering to a small group similar to the group you will give the test to. Standardized and Researcher-made may be (1) norm-referenced or (2) criterion referenced

6. Achievement Tests cont Norm-referenced tests: Thesetypes of test enable the researcher to compare the individuals’ performance on the tests to the performance of other individuals (e.g., use of percentiles). Items will yield a wide range of scores. Performance of individual is interpreted in terms of relative position in a specified reference group. Criterion-referenced tests: This type of test enables researchers to describe what a specific individual can do, without reference to the performance of others. Performance is reported in terms of the level of mastery of some defined content or skill domain – usually report the percentage of items answered correctly – important to have a representative set of items

7. Achievement Tests cont • Test Performance Range • Ceiling Effect – a test with a low ceiling is one that is too easy for some of the subjects to whom it was administered • Floor Effect – this is influenced by the difficulty of the test – you cannot get real evidence about how subjects could perform because the difficulty of the test itself, rather than subjects’ ability, determines their performance.

8. Performance Assessments • Performance Assessments – focus on what an individual can do rather than what an individual knows – driver’s license test, portfolio • Construction of performance assessments: • Clearly state the object • Problems or exercises that give students an opportunity to perform a simulation or actual task • All perform the same task • Develop an instrument that lists the relevant criteria • (Time consuming so can be more expensive)

9. Aptitude Tests • Aptitude tests – measure general ability; used to be referred to as intelligence tests, but no longer because of disagreement over the definition of intelligence – often referred to a scholastic aptitude tests • Example - SAT or the Scholastic Aptitude Test and GRE or Graduate Record Exam

10. Aptitude Tests cont • Individual Aptitude Tests – must be given by a trained psychometrician to an individual subject • Standford Binet – verbal, quantitative, abstract/visual reasoning, and short-term memory and overall IQ • Wechsler tests – divided into subtests and two scores for each person – verbal IQ and nonverbal IQ and overall IQ • Wechsler Preschool and Primary Scale of Intelligence-Revised – 4 to 6 1/2 years • Wechsler Intelligence Scale for Children – Third Edition • Wechsler Adult Intelligence Scale

11. Aptitude Tests cont • Group Aptitude Tests • Tests of Cognitive Skills • Cognitive Abilities Test Testing is moving from paper and pencil tests to computer-based testing.

12. Measures of Personality • Measures of Personality • Inventories – collection of statements describing behavior patterns. Check “yes” or “no” or “uncertain” to indicate if the statement is characteristic of you. Compute scores by counting the agree statements that coincide with what the researcher is trying to measure. An example of a self-report inventory that measures 1 trait would be California F-Scale (authoritarianism) and Myers-Briggs Type Indicator measures several traits

13. Measure of Personality cont • Projective Techniques – these type of techniques ask the individual to respond to ambiguous or unstructured stimulus. Person is expected to project into the stimulus his or her own needs, wants, fears, anxieties. Used by clinical psychologists for studying people with emotional problems. Problems with validity. Examples include: Rorschach Inkblot Technique or Thematic Apperception Test (pictures and need to tell story about it)

14. Attitude Scales An attitude may be defined as a positive or negative affect toward a particular group, institution, concept, or social object Scale is a set of categories or numeric values assigned to individuals, objects, or behaviors for the purpose of measuring variables; use a scale to measure the attitudes of mothers toward inclusion for example. Attitude scales use multiple responses – responses from questions – and combine the responses into a single scale score. Think of subtopics relating to the attitude and then write items on each subtopic

15. Attitude Scales cont • Attitude Scales – summated rating scales – half items positive and half items negative – representative sample of all possible opinions on a subject – statements arranged in random order; respondents indicate if they: • Strongly agree /agree/ undecided/ disagree /strongly disagree