1 / 53

Online Measurement of Academic Programme Preferences (MAPP) for Distance Learners in Hong Kong

Online Measurement of Academic Programme Preferences (MAPP) for Distance Learners in Hong Kong. Wei-yuan ZHANG & Lettice AU YEUNG The Open University of Hong Kong.

fox
Download Presentation

Online Measurement of Academic Programme Preferences (MAPP) for Distance Learners in Hong Kong

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Online Measurement of Academic Programme Preferences (MAPP) for Distance Learners in Hong Kong Wei-yuan ZHANG & Lettice AU YEUNG The Open University of Hong Kong

  2. Project Team MembersDr. Wei-yuan ZHANG Dr. David MURPHYMs. Sylvia HUI Ms. Annie CHAN Ms. Lettice AU YEUNG Ms. Elaine KWOKMr. Albert TSE (IT Support)Mr. Henry LIN (IT Support) Funded by the PACRD The Open University of Hong Kong

  3. Advisory GroupsThe National Career Development Association (USA)The National Career Guidance Association (Mainland China)Department of Psychology and SociologyNapier University (UK)

  4. Introduction

  5. Background • Needs for life-long education • Rapid development of ODL • Increase in ODL programmes • OUHK: 114 programmes

  6. Problems How to help potential distance learners understand: • their psychological characteristics • programme requirements • make wise academic choices

  7. A New Issue Providing distance guidance on programme preferences for distance learners

  8. Literature Review Students’ needs • Understand programme preferences • Programme selection guidance Employers’ requirements • Understand interests and job requirements • Good psychological preparation

  9. Literature Review Method: Psychological testing UK: JIIG-CAL (Job Ideas and Information Generator - Computer Assisted Learning); CENTIGRADE USA & Canada: SDS (Self-directed Search)

  10. Literature Review Existing academic preferences tests • Limited to Western countries • Charge high fees

  11. Literature Review Differences in: • Socio-economic structure • Cultural background • Educational system • Family dynamics • Linguistic considerations

  12. Purpose To develop the first online self-directed standardized test on the measurement of academic preferences (MAPP) for potential distance learners at the OUHK

  13. Value Expected to: • Enhance students’ satisfaction with programme selection • Possibly reduce the rate of student attrition

  14. Theoretical Framework

  15. Trait-factor theory Three steps: • Trait: individual • Factor: distance education study programmes • Relationship between the two

  16. Design Step 1: Identify personal attributes Step 2: Establish minimum requirements for DE study programmes Step 3: Design an online matching computer programme

  17. Methodology

  18. Method Psychological testing

  19. Instrument • Initial pool: 330 items • Related school, social, family and leisure activities, etc. • Three parts: interests, abilities and temperaments

  20. Sampling Technique Multi-stage stratified cluster sampling technique: • Geographical distribution • School academic levels • Gender

  21. Sample Age of sample: 17 –19 • Age 15: preferences stabilized • Age 17: preferences established Aiken (1997) Psychological Testing and Assessment, 9th Edition, p. 234. • OUHK programmes: 17 years old and above

  22. Procedures

  23. Step 1: Designing draft instrument • 330 items (school, social, family and leisure activities) in the Hong Kong context • Reviewed by team members

  24. Step 2: Validating draft instrument • Item validation by 49 psychologists/ career counsellors • Results: 66 items were deleted

  25. Step 3: Pilot study • 264 remaining items were randomized • Sample: 13 students (Forms 4 and 5) • Minor changes in language made

  26. Step 4: 1st survey: Test-retest • Sample size: 54 students • Time interval: 4 weeks • Reliability coefficient: 0.67 • 33 items were further discarded with item-total correlation < 0.4 • Remaining 231 items • Reliability coefficient: 0.72

  27. Step 5: 2nd survey: Identifying personal attributes • Sample: 1,217 students • Factor analysis • 8 interpretable factors (attributes) were identified

  28. Step 6: 3rd survey: Verification of 8 attributes • Sample size: 764 students • Factor analysis • Same 8 attributes identified • 132 items retained • Alpha reliability: 0.962

  29. Step 7: Determining minimum desirability ratings • 42 Course Coordinators • 115 OUHK programmes • Minimum entry requirements • Based on 8 personal attributes • Method: Job analysis

  30. Step 8: Developing the Online MAPP • Java programming language • UNIX (Solaris) Platform • Oracle 8i database • Data exported into Excel format

  31. Results

  32. The MAPP Inventory 132 items, 8 personal attributes (Accounting for 41.6% of the total variances, P<.05)

  33. Attribute Definition Influential An individual who possesses leadership qualities is well rounded, confident, organized and perceived as a role model among his/her colleagues. This individual is also articulate and persuasive and works well with people. This individual is creative and analytical and thus adaptable to changing conditions. Social This individual is interested in how people communicate and function as a society. He/she is well read and critical of the information he/she gathers. Staying current and being well informed in political and social issues also characterizes this individual. This individual engages with others in social and political activities. Helping This individual’s main interest is in helping people. He/she possesses good oral, communication and conflict resolution skills and is confident in his/her abilities. This individual is open-minded and understanding and works well with others. Explorative This individual is interested in science and engineering. He/she possesses skills to manipulate objects and perform experiments. This individual possesses strong analytical skills. Such an individual has a preference to work in a laboratory. Technological This individual is interested in machinery and electronics. He/she has strong hands on skills to fix and manipulate machinery, specifically computers. This individual’s computer skills are analytical, creative and intuitive, being able to develop programs and solve technical problems. This individual works independently. Logical This individual is interested in numbers and problem solving. He/she examines problems in abstract ways and performs tasks systematically. He/she enjoys solving mathematical games with others. This individual is rational and would be a valuable team member in group situation. Reflective This individual has interests in literature and linguistics. He/she is reflective and analytical about what he/she reads and has a broad vocabulary. This individual enjoys the challenges of learning new skills in reading, writing and speaking. This individual is patient and focused in his/her work. Enterprising This individual is attracted to a career in business. He/she is keen to study and acquire knowledge in business related disciplines such as administration and management. This individual also has a good business sense and stays current with news in the business world. He/she has a goal to succeed and is well read on business people who have achieved such goals. Personal Attributes

  34. Factor Name No. of Items Cronbach’s Alpha Split-Half Reliability Influential 22 .92 .92 Social 13 .90 .86 Helping 21 .91 .89 Explorative 20 .93 .92 Technological 15 .93 .87 Logical 14 .93 .87 Reflective 18 .90 .84 Enterprising 9 .90 .82 Reliability Table 3: Subscale internal consistencies

  35. Reliability (summary) • From Cronbach’s Alpha • The internal consistency of the total score of MAPP was 0.962; • Subscale internal consistency coefficients ranged from 0.90 to 0.93. • Split-half reliability • Split-half reliability coefficient of the MAPP scale was 0.92 • Subscale split-half correlations ranged from 0.82 to 0.92

  36. Validity • Content validity - items evaluated by 49 experts • Construct validity - established through a series of factor analysis

  37. Influential Social Helping Explorative Technological Logical Reflective Social .46 Helping .63 .36 Explorative .22 .15 .15 Technological .23 .16 .11 .46 Logical .23 .11 .13 .57 .44 Reflective .40 .51 .35 .08 .09 .03 Enterprising .34 .66 .24 .07 .17 .17 .29 Sub-scale inter-correlations Table 4: Sub-scale inter-correlations

  38. Sub-scale inter-correlations • Summary • Correlation coefficients ranged from 0.03 to 0.66 • There was no indication of multicollinearity

  39. Type Male (N=958) Female (N=966) P-value Mean SD Mean SD Influential 3.24 .57 3.19 .52 0.24 Social 2.74 .72 2.71 .66 0.31 Helping 3.42 .56 3.57 .51 0.00** Explorative 3.06 .75 2.66 .72 0.00** Technological 3.05 .79 2.52 .66 0.00** Logical 2.98 .75 2.64 .72 0.00** Reflective 2.91 .65 3.06 .63 0.00** Enterprising 2.64 .83 2.60 .75 0.30 Standardization Table 5: Gender differences in subscale mean scores ****Significant at the (P<0.01) level

  40. Standardization (Summary) Attributes without Gender differences • Influential , Social and Enterprising Attributes with Gender differences • Males: Explorative, Technological, and Logical • Females: Helping and Reflective

  41. Online MAPP To MAPP

  42. Auto-data Collection Excel format • For further validation & to automatically update gender-based norms

  43. Student Evaluation Sample: 227 students Degree of usefulness • Very useful/ useful: 50.2% • Neutral: 45.4% • Not useful/not useful at all: 4.4%

  44. Student Evaluation Degree of satisfaction • Very satisfied/ satisfied: 56.7% • Neutral: 37.9% • Not satisfied /not satisfied at all: 5.4%

  45. The MAPP could help me in Strongly agree/ agree Neutral Mean SD N Understanding my psychological characteristics 141 (62.1%) 80 (35.2%) 6 (2.6%) 3.64 .638 227 Understanding my study programme preferences 138 (60.8%) 76 (33.5%) 13 (5.7%) 3.62 .715 227 Disagree / strongly disagree Widening my considerations of programme choices 119 (52.4%) 88 (38.8%) 29 (8.8%) 3.52 .806 226 Planning my future educational direction 115 (50.7%) 98 (43.2%) 14 (6.2%) 3.47 .712 227 Making wise educational decisions 95 (41.9%) 110 (48.5%) 22 (9.7%) 3.35 .752 227 Student Evaluation

  46. CCs’ Evaluation Positive responses (10 CCs) “It is a very useful tool, which provides an indication of which OUHK programmes could be considered”. “You have taken the proper steps to validate the constructs. I think the instrument will be useful to individuals considering which programme to go into”.

  47. CCs’ Evaluation Negative Responses (2 CCs) “Students would have some idea on what to study before they use this system”. “Most prospective students would not have the patience to go through the MAPP carefully. How should students choose when the system recommends more than one programme?”

  48. Discussion and Conclusion

  49. Gender differences in personal attributes • May be influenced by different experiences and Chinese culture

  50. Improving distance guidance to support ODL • Study satisfaction • Potential reduction of attrition

More Related