1 / 36

Developing Testing

Rajiv Unnikrishnan - Business Head - SkillProfiler. Developing Testing . www.skillprofiler.com rajiv@skillprofiler.com. Content preparation Choice of authoring tool Scoring logic and methodology Report generation Effective usage of test results. Developing a test . CONTENT

louvain
Download Presentation

Developing Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rajiv Unnikrishnan - Business Head - SkillProfiler • Developing Testing www.skillprofiler.com rajiv@skillprofiler.com

  2. Content preparation Choice of authoring tool Scoring logic and methodology Report generation Effective usage of test results Developing a test

  3. CONTENT PREPARATION

  4. Identify the most critical skills Break them into sub skills Create questions for each sub skill Ascertain the assessment method… Contents identification Oracle 9i Adaptive

  5. It is an optimal method for soliciting and analyzing feedback and for adjusting our actions based upon the feedback. Adaptive Methodology

  6. Can be compared to a High Jump event Bar height (question difficulty) is continuously adjusted based upon prior jumps (answers) until the maximum cleared height (knowledge) is identified. Adaptive Model

  7. You have used Adaptive Model even as a kid Guess My Number Game Objective: To guess the secret number in as few guesses as possible • selects a secret number from 1 to 100 • Player 2 makes a guess at the secret number • Player 1 provides FEEDBACK response of ‘higher’ or ‘lower’ • Play continues until Player 2 guesses the secret number.

  8. Non-Adaptive Strategy Hit or Miss Method Slow but Steady Method I Have No Clue Method • Is the secret number 1? • -higher • Is the secret number 2? • -higher • Is the secret number 3? • -higher • Is the secret number 4? • -higher • Is the secret number 5? • -higher • Is the secret number 6? • -higher • Is the secret number 7? • -higher • Is the secret number 8? • higher • Is the secret number 9? • -higher • Is the secret number 10? • -higher • Is the secret number 11? • -higher Is the secret number 60? -higher Is the secret number 40? -I said it was higher than 60! Is the secret number 30? -What are you, an idiot? Is the secret number 20? -Let’s play Monopoly Is the secret number 25? -higher Is the secret number 80? -lower Is the secret number 26? -higher Is the secret number 30? -higher Is the secret number 70? -lower Is the secret number 35? -higher Is the secret number 68? -lower Is the secret number 67? -lower Is the secret number 39? -higher Is the secret number 61? -lower Is the secret number 60? -lower

  9. 50 Is the secret number higher Is the secret number 75 lower Is the secret number 62 lower Is the secret number 56 higher 59 Is the secret number lower Is the secret number 58 The Adaptive Model Model: Each guess adapts to the feedback of prior guesses Strategy: Maximize value of feedback YES

  10. Secure Random selection of questions from the appropriate difficulty pool. Knowledge of answers results in more difficult follow-up questions Efficient No time is wasted on questions that are too hard or too easy. E = MC 2 Benefits of Adaptive… Hey Albert, How Much is 2 + 2 ? IT Testing - USP

  11. Choice 0f Authoring Tool

  12. A software environment used for developing, implementing and deploying assessment contents. At this stage the need is for a tool to translate our design and prepared content into the requirement we envisioned Authoring Tool

  13. Degree of Difficulty 6.7 Easy You’ve got to be kidding! Medium Hard Very Easy Adaptive System Driver… IT Testing - USP

  14. Online Web Based Online LAN Based Online PC Based Offline Based Seamless Technology…

  15. SCORING LOGIC & METHODOLOGY

  16. PARADIGM SHIFT IN SCORING Replace Binary Feedback (Correct / Wrong) with Multi-Dimensional feedback (Absolutely Correct, Somewhat Correct, partially correct, reasonably wrong, wrong, not even close)

  17. BRINGING ACCURACY TO SCORING So, how do we get multi-dimensional feedback? • Multiple Correct Responses • Multiple choice questions with 1-3 correct answers out of 5 possible answers • Each answered weighted for degree of accuracy or inaccuracy (perfectly correct, partially correct, wrong, not even close) 25 Answer Combinations A ABD CE B ABE DE C ACE ADE D ACD BDE E BC CDE AB BD AC BE AD BCD AE BCE ABC CD Each answer combination • identifies a different level of knowledge • has its score based upon contained answers • leads to a different level of follow-up question

  18. Which of the following are states in the United States of America? a. Texas b. Paris c. California d. Washington, D.C. e. Jupiter Sample Adaptive Model Question Subject: U.S. Geography

  19. Sample Adaptive Model Question Answer Combinations Reflect Partial Understanding California and Texas Full credit answer Leads to most difficult follow-up question Credit for selecting Texas, loss of credit for not selecting California Texas Demonstrates no understanding of US geography; the test taker is most likely guessing Jupiter Not Even Close answer; loses most credit and leads to easiest follow-up question Paris, Jupiter and Washington D.C. Less wrong than Texas and Paris; reflects partial understanding of US geography Texas and Washington D.C.

  20. Detailed Feedback based Reporting We assign the ‘secret number’ in each sub-skill a knowledge level designation

  21. REPORT GENERATION…

  22. ORACLE9i DATABASE ADMINISTRATION - Complete Assessment Results ORACLE9i DATABASE ADMINISTRATION - Complete

  23. Applying The Test…

  24. Internal Certification Skill Assessment Model Beneficiaries are HR Skill-wise Competency Mapping Skill Inventory Training & Development Skill-wise Segmentation Recruit Assess & train on skill gap areas Hire the best talent Beneficiaries are Business vertical

  25. Custom Learning Perspective

  26. GREEN BAR – Portion of group showing strength in the sub-topic RED BAR – Portion of group showing weakness in the sub-topic YELLOW BAR – Portion of group showing proficiency in the sub-topic Applying The Adaptive Model Group Knowledge Audits ORACLE9i PL/SQL PROGRAMMING

  27. Emphasize these sub-topics Remove these sub-topics Applying The Adaptive Model Customize Classroom Training ORACLE9i PL/SQL PROGRAMMING

  28. Applying The Adaptive Model Build Homogeneous Classes Class C only requires training in 5 of the 12 defined skills for this curriculum.

  29. Proposed project team is less than ideal. 90% of team is weak in this critical sub-topic and the team lacks an expert to mentor them (no green bar). Restructuring of team is indicated. Applying The Adaptive Model Skills Management After training, build optimum project teams Sub-topics listed in order of weakest to strongest

  30. Skill Inventory… a click away Java Talent pool

  31. Applying The Adaptive Model After Training Certification • Incentive for training • Validate training • Career advancement • Establish criteria for project participation • Identify mentors

  32. Applying The Adaptive Model Certification • External Certification • Provided by outside 3rd party does not reflect client specific knowledge sets • Clients concerned External Certification makes employees more marketablePercentiles • Relative measure that changes with the knowledge of other individuals rather than with the knowledge of the test taker. Raw Scores • Requires long calibration study to fixed a knowledge level (how much knowledge does a score of X represent) and must be normalized across multiple test subjects (does a score of X indicate the same level of knowledge in all test subjects?) • Reflects an aggregate across all sub-skills. (Do two individuals with the same raw score have the same set of skills?)

  33. Applying The Adaptive Model • Capitalize on the Adaptive Method’s unique ability to qualify sub-skills Assessment is broken down into sub-skills that collectively cover the entire scope of the subject Each sub-skill is classified as a weakness, proficiency or strength Sub-Skills are group by knowledge level Establish certification criteria based upon the extent of subject coverage • Percent of sub-skills that are not weak • Percent of sub-skills that are strong

  34. Certification Process Assign weights to each knowledge level Assign weights to each sub-skill reflecting the specifics of your environment. • Arrive at level wise and measure each one against the specified skill • Certify based on knowledge & skill levels

  35. Determine the most critical skill Break them into sub skills Create questions based on adaptive (difficulty based) Choose the right authoring tool Tests should be scalable technologically Report generated should be elaborate Focus to be on Skill Assessment Model (SAM) Summary

  36. Thank you rajiv@skillprofiler.com Online Skill Assessment Solutions

More Related