1 / 15

Welcome to the CIRTL Network’s Teaching-as-Research Capstone Seminar

Survey, Data Collection, Instruments and Questionnaires. Welcome to the CIRTL Network’s Teaching-as-Research Capstone Seminar. Dr. Mary Elizabeth Besterfield - Sacre. Session begins at 3 -4 ET/2-3 CT/1-2 MT/12-1 PT  When you join the room please run the Audio Setup

carlow
Download Presentation

Welcome to the CIRTL Network’s Teaching-as-Research Capstone Seminar

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Survey, Data Collection, Instruments and Questionnaires Welcome to the CIRTL Network’sTeaching-as-Research Capstone Seminar Dr. Mary Elizabeth Besterfield-Sacre • Session begins at 3-4 ET/2-3 CT/1-2 MT/12-1 PT  • When you join the room please run the Audio Setup • Wizard: Tools Menu->Audio->Audio Setup Wizard • While we wait for the session to begin, please feel free to • test your microphone and webcam • If you are experiencing problems and/or have questions, • please type into the chat window Director, EERC Associate Professor and Fulton C. Noss Faculty Fellow Department of Industrial Engineering University of Pittsburgh

  2. Ways to Interact during theTAR Working Session • Turn on/off your microphone: • Raise your hand if you have a question or comment • Turn on/off your video: • Use the chat window to add comments, ask questions, or request help

  3. Survey, Data Collection Instruments, Questionnaires Mary Besterfield –Sacre Director, EERC Associate Professor and Fulton C. Noss Faculty Fellow Department of Industrial Engineering University of Pittsburgh

  4. Once Upon A Time… • I was a grad student… • My research focus • Industrial statistics • Quality assurance • Finished one NSF grant in healthcare and put on another • Engineering education • Improving the freshman engineering curriculum – integrated curriculum • I pouted…

  5. Integration of my skills to solve an engineering education problem • The effort needed a means to determine if change had occurred in students • Exam questions • Projects • Attitudes and perceptions of engineering • Retention in engineering • I set up my first evaluation plan… • Objectives, strategies, measures of the objectives

  6. Thought it would take a weekend… • First researched the area of student attitudes and students leaving the STEM field • Seymour and Hewitt – Talking About Leaving • Others in the STEM areas • Search for other instruments • None existed • Got a great survey book - Dillman • Developed an initial survey • Some closed ended responses based on the research, but two critical open-ended responses • Things that enticed students about engineering • Things that turned students off from engineering • Piloted the 1st survey on Soph, Jr and Sr Content Validity

  7. Thought it would take a weekend… • Initial data analysis • Reliabilityanalysis – Crombach’s alpha • Exploratory factor analysis – some hypotheses yes; others no • Grounded theory approach to open-ended questions – created new items • Revised the questionnaire • Conducted verbal protocols or cognitive interviews • Revised the questionnaire – again • Obtained my first IRB Human Subjects clearance

  8. Thought it would take a weekend… • Launched it on Freshman engineering class • “Old” programming • Launched it on incoming Freshman engineering class • Incoming pre attitudes about engineering • Again, at the end of their Freshman year • “New” programming

  9. Thought it would take a weekend… • Data analysis • Confirmatory factor analysis • Establishment of weights based on the factor analysis • Item reliability analysis - again

  10. What did we learn? • Differences between the two cohorts of freshmen • Besterfield-Sacre, ME, Atman, CJ, and Shuman, LJ. “Engineering Student Attitudes Assessment.” Journal of Engineering Education 87, no 2 (April 1998): 133-141. • Prediction of freshmen who will leave engineering • Besterfield-Sacre, M.E., C.J. Atman and L.J. Shuman. “Characteristics of Freshman Engineering Students: Models for Determining Student Attrition and Success in Engineering.” The Journal of Engineering Education 86, no 2 (April 1997): 139-149. • My first grant • Besterfield-Sacre, M.E., M. Moreno, L.J. Shuman, and C.J. Atman. “Gender and Ethnicity Differences in Freshmen Engineering Student Attitudes: A Cross-Institutional Study.” Journal of Engineering Education90, no 4 (October 2001): 477 - 489. Criterion-related Validity - Concurrent Validity Criterion-related Validity – Predictive Validity Criterion-related Validity - Concurrent Validity

  11. Since then… • People still ask for it! • I get about 5-8 schools ask per year • A group at ASU redid the work a few years back to ‘update’ the instrument • Construct Validity? • Demonstrates an association between the survey and the prediction of a theoretical trait

  12. Your questions, issues, challenges?

  13. Thank You for Joining Today’s Session What did you think of today’s session? Please take a minute right now to fill out a 1-question survey on today’s experience: We very much appreciate your feedback.

  14. Working Session: Qualitative and Quantitative Data Management and Analysis March 17, 2014 3-4 ET/2-3 CT/1-2 MT/12-1 PT Facilitated by: Mark Urban-Lurain, Associate Professor, Center for Engineering Education Research, Michigan State University Upcoming TAR Capstone Session To sign up to hear about these and other CIRTL events, emailinfo@cirtl.net.

More Related