760 likes | 888 Views
Assessments Through The Learning Process. October 2013 Eric Shepherd. Questionmark , CEO eric@questionmark.com www.questionmark.com. Thank you. Using the cards. C. A. D. B. You have colored cards on your chairs/tables Get one of each color, lettered A, B, C and D
E N D
Assessments Through The Learning Process October 2013Eric Shepherd. Questionmark, CEOeric@questionmark.comwww.questionmark.com
Using the cards C A D B • You have colored cards on your chairs/tables • Get one of each color, lettered A, B, C and D • Let’s practice holding up cards • During this presentation we’ll be using questions to show you the power of questions • Please answer quickly and don’t be embarrassed – we are here to learn!
Research Suggests… A B C D • Learners generally prefer response cards to simple hand-raising • Response Cards Shown to Improve • classroom engagement • amount they learn • amount they retain -- Dr. Will Thalheimer, citing multiple examples of research on the use of response cards
How much tea was dumped into Boston Harbor at the Boston Tea Party? A B C D 100 chests of tea 34 chests of tea 342 chests of tea 342 tea bags
Answer: C • 342 chests of tea • Over 46 tons of tea • Worth (at the time) over £10,000 • £678,827 using the retail price index • £866,694 using the GDP deflator • £7,422,564 using average earnings • £10,025,165 using per capita GDP • £25,350,172 using the GDP Citation: Lawrence H. Officer, "Five Ways to Compute the Relative Value of a UK Pound Amount, 1830 - 2005" MeasuringWorth.Com, 2006.
Learning Without Questions zzzzzzzzzz Content zzzzzzzzzz zzzzzzzzzz zzzzzzzzzz zzzzzzzzzz
Learning Benefits of Questions Content Questions
We are using answer cards to… A B C D …expose poor presentation skills …stimulate thinking and keep you awake …frustrate our color blind friends All of the above
Questionmark • Provide Assessment Management Systems • To create, deliver and report on assessments • For exams, tests, quizzes, and surveys • More than 1,500 customers have purchased Questionmark • Corporate, Government, and Education markets • More than 14,000 authoring systems installed • Last month users created 40,000+ questions in Questionmark • Offices in US, UK, Canada, Germany, Belgium • Available in numerous languages
Sample List of Canadian Customers • Loyalist College • Manitoba Hydro • Memorial University of Newfoundland • Moneris Solutions • National Association of Pharmacy Regulatory Authorities • Northern Ontario School of Medicine • Omnivex Corporation • Plains Midstream • Safety Codes Council • Scotiabank • Telus : Telus Supporting Units • University of Calgary : Faculty of Veterinary Medicine - University of Calgary • University of Manitoba - Faculty of Medicine • University of Ottawa • University of Ottawa - Faculty of Medicine • University of Saskatchewan • Alberta Health Services • Aluminerie Alouette Inc. • Apache Corporation • ATB Financial • British Columbia Hydro • Canada Border Services Agency - CBSA • CMLTO - College of Medical Laboratory Technologists of Ontario • Department of National Defence • Everest Colleges Canada Inc • ExxonMobil • Flakeboard Company Limited • Fortis BC Inc • Grant MacEwan University • International Competency Assessment Board • Invesco Canada Ltd. • Justice Institute of British Columbia
Financial Services Pharma / Health High Tech / Manufacturing Telcom Energy / Utilities Public Sector Education Transportation Prof. Services Retail Who Else Uses Questionmark?
What would you like from this session? A B C D …brain dump; tell me everything …I don’t know; say something interesting …hurry up; I am only here for the networking …why so many questions?
Increase Revenue = € £ $ ¥ Reduce Costs = € £ $ ¥ Reduce Time to Market Things that Drive What-To-Do
Health &Safety Money = € £ $ ¥ Laws People or Talent How Long Do We Have? Resources Practical Limitations for Planning
What are the business drivers that drive the use of learning and assessments? A B C D Increase revenue, time-to-market, laws Increase revenue, reduce costs, safety Increase revenue, reduce costs, time-to-market Money, time-to-market, laws and talent
What are the practical limitations of using assessments? A B C D Money, health and safety Money, laws, health and safety Money, laws, resources, and safety Money, laws, time-to-market, health and safety and, access to talent and resources
What To Do How To Do It Who Will Do It Knowledge, Skills and Abilities Required What Will They Do What Will They Use
Competencies What To Do How To Do It Who Will Do It Knowledge, Skills and Abilities Required What Will They Do What Will They Use
Mental Check to Confirm a Competency “Given What Do What”
Competency is the “Ability To Do”(or “Ability To Perform”) and requires…. Cognitive Skills
Example - Attitudes and Behaviors • Customer Results • Consistently meets Customer Needs. Provides solutions for external and internal customers that meet and/or exceeds customer expectations. Identifies opportunities that will benefit the customer and works in partnership with the customer to drive the business forward. • Results Oriented • Ability to meet and exceed goals, make difficult decisions, act on opportunities that create value for the company. • Demonstrate ability to exceed stated job responsibilities, take informed risks, make principled decisions, and faces high-stakes issues head on. • Create a work environment where others feel it is safe to take risks.
Competencies Required Knowledge, Skills and Abilities Required
Knowledge, Skills and Abilities Required Assessment • How do we know the Knowledge, Skills and Abilities required? • Job Task Analysis • Survey people who do the job • MoSCoW analysis • Knowledge, Skills and Abilities that we: • Must have • Should have • Could have • Would be nice to have
Knowledge, Skills and Abilities Required Skills Knowledge Knowledge Abilities Abilities Skills What We Need What We Have Skill Survey, Quiz, or Test, Assessment
What We Have Knowledge, Skills and Abilities Required Skills Knowledge Knowledge Abilities Abilities Skills What We Need Topics To Train Knowledge - = Skills Abilities
Which is the best answer to the question… Why are skill surveys useful? A B C D Helps us survey skills Helps us know what actions to take Know what knowledge we have Determine qualified candidates
Assessment Thru Learning Process Learning Work Place Mentoring Collaborative & E-learning Instructor In/As LearningAssessments Prepare For Learning Of-LearningAssessments Evidence Accumulated
Assessment Thru Learning Process Learning Work Place Mentoring Collaborative & E-learning Instructor Job Task Analysis Needs Analysis In/As LearningAssessments Prepare For Learning Slow Forgetting Curve Of LearningAssessments Course Evaluations Evidence Accumulated
Pre-Learning Assessments • Purpose: • Create intrigue • Route to course • Inform instructors • Establish benchmarks Learning Work Place Mentoring Collaborative & E-learning Instructor In/As LearningAssessments Prepare For Learning Of LearningAssessments Evidence Accumulated
In-Learning Assessments • Purpose: • Encourages focus on learning • Provide instant feedback to instructor • Stimulate and strengthen memory • Correct misconceptions Learning Work Place Mentoring Collaborative & E-learning Instructor In/As LearningAssessments Prepare For Learning Of LearningAssessments Evidence Accumulated
Post-Learning Assessments • Purpose: • Was learning environment okay? • Knowledge transfer occurred? • Reduce Forgetting Curve • Certifications Learning Work Place Mentoring Collaborative & E-learning Instructor In/As LearningAssessments Prepare For Learning Of LearningAssessments Evidence Accumulated
Learning Curve • Purpose: • Content repetition • Memory retrieval practice • Strengthens memory recall Ouch! The Forgetting Curve Questions Questions Knowledge/Skills Learning Experience Learning Experience Learning Experience Time
Slow the Forgetting Curve • Purpose: • Memory retrieval practice • Strengthens memory recall • Reduce Forgetting Curve Questions Questions Questions Questions Knowledge/Skills Learning Experience Learning Experience Learning Experience Time
Questions Thru the Learning Process Reduce Forgetting Create Intrigue Questions Questions Questions Questions Questions Strengthen Memory Knowledge/Skills Time
Which of the following combination of instructional activities will increase learning and reduce forgetting?Question borrowed from the work of Dr. Will Thalheimer A B C 30 minutes of instruction and 30 minutes of questioning 30 minutes of instruction and 30 minutes of review and study time Both will produce similar results
0 Dr. Will Thalheimer research:
Which of the following combination of instructional activities will increase learning and reduce forgetting?Question borrowed from the work of Dr. Will Thalheimer A B C 30 minutes of instruction and 30 minutes of questioning 30 minutes of instruction and 30 minutes of review and study time Both will produce similar results
Tests and Exams That Measure • Regulatory • Governments • Health and Safety • Examples • Driving Tests • Pilots Exams • Doctor Certification • Non-Regulatory, certification • Vendor Specific • Microsoft • Oracle • SAP • Vendor Neutral • ECDL/ICDL • Regulatory Compliance • Demonstrating compliance to government • Health/Safety/Environmental • Financial Services
Setting a Pass/Fail score? Norm Referenced (pass/fail determined after test) Criterion Referenced (pass/fail score determined before test) 250 people 250 people 3rdgroup of People 75 pass 1stgroup of people 65 pass Only 50 people can pass 2ndgroup of people 45 pass
Are Regulatory Compliance tests Norm or Criterion referenced? A B C D Norm referenced Criterion referenced Neither of A nor B Both A and B