The learning outcomes of an online reflective journal in engineering

1. The learning outcomes of an online reflective journal in engineering Stuart Palmer, Dale Holt and Sharyn Bray Institute of Teaching and Learning Deakin University

2. Introduction This presentation: • explores the value of reflective journaling in engineering education • presents a case study of the introduction of an online reflective journal • summarises the results of the students’ initial perceptions of the online reflective journal, & • investigates the contribution of the online reflective journal to unit learning outcomes

3. Reflection and reflective journals In the context of learning, reflection refers to the active intellectual monitoring and evaluation of one’s own formal learning and professional practice activities, to examine them for new understandings that add to the individual’s accumulated knowledge and experience Reflective thinking based on experiential learning is a key skill required for the lifelong learner and the socially mature professional

4. Reflection and reflective journals The use of a reflective learning journal (due to the requirement to transfer thought processes into words) is thought to be a valuable tool in developing self- and critical reflection in students For the practicing professional, the use of a work journal offers additional benefits

5. Source: http://legal.european-patent-office.org/dg3/biblio/t930039ex1.htm

6. Ex-Air Force Official Gets Prison Time Boeing Received Special Treatment in Procurement By Renae Merle and Jerry Markon Washington Post Staff Writers Saturday, October 2, 2004; Page A01 A former high-ranking Air Force procurement official was sentenced to nine months in federal prison yesterday after admitting that she approved excessive prices on contracts awarded to Boeing Co. to enhance her job prospects with the company. Conceding that she lied to prosecutors, Darleen A. Druyun, 56, revealed that she committed the Air Force to buy 100 airplanes from Boeing at an inflated price of about $20 billion as a "parting gift" before her Pentagon retirement to ingratiate herself with her future employer. She also slipped to Boeing proprietary pricing information from a rival European bidder on the aircraft contract. Druyun awarded Boeing an unrelated $4 billion contract because she felt in debt to the company for hiring her daughter and future son-in-law, according to court documents. An "objective selection" process, she said, may not have picked Boeing from the four competitors. In a quivering voice, Druyun apologized before U.S. District Judge T.S. Ellis III in Alexandria, telling the court she felt "shame and remorse" that her 30-year tenure as a government employee "has been tarnished. . . . I understand that this was wrong and I regret any damage my conduct may have caused to the Air Force." Druyun's case is the highest-profile defense procurement scandal since the Operation Ill Wind investigation, which resulted in more than 60 convictions starting in the late 1980s. It is expected to ripple throughout the industry, renewing concern about the potential pitfalls of the revolving door between government and the defense industry. Chicago-based Boeing, the Pentagon's second-largest contractor, will likely face fresh questions about several of its contracts, and the procurement system that allowed Druyun to favor one company over another will come under sharper scrutiny. Druyun, a civilian, was at the grade of a lieutenant general when she retired and became vice president in charge of Boeing's missile defense systems in January 2003. Druyun's Boeing salary -- $250,000 plus a $50,000 signing bonus -- was nearly double the top Pentagon pay for her position. Officials said Druyun admitted the extent of her deceptions only after being subjected to a polygraph test. She acknowledged altering her personal journal before turning it over to prosecutors. "She did great harm to the government, and that harm is continuing now," said Assistant U.S. Attorney Robert W. Wiechering. Source: http://www.washingtonpost.com/ac2/wp-dyn/A64968-2004Oct1

7. Reflection and reflective journals It is recognised that reflection is not only an individual activity, but may contain social/learner-learner aspects as well It is reported that collaborative reflective activity and the ability to compare one’s own thinking with that of other learners yields positive results and better facilitated learning than individual reflection

8. The context SEB421 Strategic Issues in Engineering This unit consisted of three modules: • Technological Forecasting and Assessment; • Policy Design in Engineering Organisations; • Issues in Productivity Improvement.

9. SEB421 student locations Source: http://commons.wikimedia.org/wiki/Image:Welt_Mercator_Atlantik.png

10. A reflective journal exercise At the completion of the weekly class, students were asked to respond in writing in their journal to the following two questions: • “What did I learn today?”, and • “How will this be of use to me in the future?” Off-campus students completed their journal across the semester and submitted their written journal at the end of the semester

11. A reflective journal exercise Issues On-campus: The journal took up a significant amount of class time Off-campus: Some journals were obviously produced at the end of semester, rather than across the semester

12. An online reflective journal Students were asked to make their weekly journal entry in an online discussion area A separate discussion sub-area was created for each nominal class week to provide some structure and direction to students, and to break the large number of student postings into manageable sections

13. An online reflective journal - issues While it is recognised that criterion referenced marking of student reflective journals may potentially constrain student expression It can also encourage greater student engagement with the intended pedagogical aims of the journal

14. An online reflective journal The online nature of the system meant that all students could post their journal entries weekly, regardless of their study location or mode of study Because the journal entries were posted to a discussion area with open read access to all class members, all journal postings were potentially available to all students to read

15. Evaluation of initial trial Questionnaire: • demographic information – age; gender; course of study; location of study; • reflective journal – did you understand its purpose?; rate its value; • use of journal – frequency of access; did you read submissions of other students?; • online system – rate the system’s ease of use; • general – what aspects of the online reflective journal were most useful?; least useful?

16. Response rate and demographics †- International exchange students and non-engineering students enrolled in a single unit only

17. Value and use of reflective journal

18. Most useful aspects of journal

19. Least useful aspects of journal

20. Contribution to learning outcomes • gender; • age; • mode of study (on- or off-campus); • weighted average mark (WAM – a proxy for general prior academic ability); • semester of offer cohort grouping; • number of reflective journal entries posted (a proxy for ‘quantity’ of engagement); • average mark per journal entry (a proxy for ‘quality’ of engagement); • number of reflective journal entries read; and • final unit mark (out of 100 – a proxy for unit learning outcome)

21. Pooled data characteristics summary

22. Regression model for final unit mark ANOVA test: F386 = 144.18, p < 2×10-62 R2 = 0.53 Std Err of model = 8.79

23. Contribution to learning outcomes • prior academic performance was an important predictor of final unit mark • the number of journal posts that a student made was an important predictor, and this was separable from WAM, and contributed at somewhat above the ‘face value’ of up to 1 mark per posting • even though mode was not significantly correlated to final unit mark on a direct pair-wise basis, after the effects of WAM and number of posts, mode became significant in accounting for some of the remaining variation in final unit mark, this being a negative influence for off-campus students

24. Conclusions / take-home messages It appears that it is the activity of formally reflecting on learning that makes a contribution to student learning outcomes, rather than the act of simply reading the posts of other students

25. Conclusions / take-home messages Although, student prior academic performance was a significant predictor of unit learning outcomes, the additional contribution made by each new reflective journal posting suggests that introducing an element of ‘compulsion’ for students to make journal postings, such as through attaching marks to journal postings, would be beneficial

26. Thank you for your time