1 / 24

June 19, 2012

Evolving Your PBT Program. The Citrix Story. Kpayah Tamba, Citrix. June 19, 2012. About Citrix. At Citrix, we continue to raise the bar. Real World Skills. Combination of Live Labs, Simulations, Board Defense, and S cenario-based Items. Simulations. Live Labs. Scenario-based Items.

ros
Download Presentation

June 19, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix June 19, 2012

  2. About Citrix

  3. At Citrix, we continue to raise the bar • Real World Skills Combination of Live Labs, Simulations, Board Defense, and Scenario-based Items Simulations Live Labs Scenario-based Items Multiple-choice Items

  4. CCIA CCEE CCAA CCA Citrix Certification Program Top 3 cert tiers consist of exams with performance based items

  5. Design Approaches at each Tier

  6. Citrix Evolution Multiple-choice based exams First simulation based exam First PBT based exam 75% of cert tracks now includes PBT exams

  7. How we evolved our program

  8. Phase I: MCQ Approach Standard multiple-choice: single response and multiple response items

  9. Phase I: MCQ Cont’d • Pros • Easier to develop • Cheaper to develop • Easier to grade • Easier to deliver • Cons • Highly vulnerable to item theft • Not the best item type for assessing performance based tasks

  10. Phase II: Live Lab Approach • Testing using the real products in a virtual environment

  11. Phase II: Live Lab Cont’d • Pros • Test takers embraced this new form of testing • Citrix was able to measure hands-on skills • The value (face validity) of our certification program increased • Cons • This new form of testing was not scalable • Candidates had to travel long distances to take the exam • The design and delivery mechanisms limited the number of exams that could be delivered at any given point • The manual scoring resulted in long wait times for results

  12. Phase III: Simulation Approach • Testing using simulated environments that look and function almost like the real products

  13. Phase III: Simulation Items Cont’d • Pros • Simulations are scalable • Simulations allow for the measurement of hands-on skills • Scoring is automated • Cons • Simulations are costly to build • Simulations take a considerable amount of time to design and develop • Our simulations are closed-ended; therefore they imposed limitations on our test takers

  14. Phase IV: Board Defense and Live Lab Items Approaches • In 2012, Citrix rolled out a six-week intensive training program geared towards certifying our top level audience • At the end of six weeks of training (eight hours a day, five days a week), candidates are required to complete a rigorous 14-hour hands-on practicum in one sitting, where they build a virtualization solution consisting of Citrix products • Candidates must also defend the solution they build before a panel of SME judges

  15. Phase IV: Board Defense and Live Lab Items Cont’d • Pros • The use of both live lab and simulation items allow for Citrix to continue to measure hands-on skills at a high level, while using two different design approaches based on our need to scale • The use of the board defense approach allows us to determine whether test takers know the “why” behind the “how” • Cons • The cost to develop simulation items is still a problem • Simulations take a considerable amount of time to design and develop • Live labs are difficult to scale • Live labs are difficult to score • A uniform board defense experience is also difficult to maintain

  16. So what’s next for us at Citrix?

  17. Next Steps • Increase our PBT footprint • Fully automate the scoring of our live lab items • Complete the development of a grading tool for our board assessment • Localize our live lab items • Localize our board assessment exam

  18. Conclusion

  19. Conclusion • As you embark on your next steps to expand your PBT program, ask yourself the following questions: • How will this design approach improve my program? • How do I start? • Do I have the budget? • Do I have the skills, and if not, can I hire or contract the right talents? • What is my end goal? • How do I get there?

  20. Q & A

  21. Group Activity • The design approach document provides: • Examples of design approaches • Characteristics of those design approaches • When to use them • When not to use them • Exercise Objective: • This exercise will help us identify solutions to potential issues that one could encounter when using a design approach • We will break up into four groups • Each group will focus on an approach • Group A – Live Environment • Group B – Board Assessments/Oral Exams • GroupC – Simulating Software Using Software • Group D – Simulating Hardware Using Software

  22. Group Activity – Cont’d • In each group, for the assigned design approach, identify solutions to the following issues: • Skills required to develop item type • Delivery challenges • Psychometric challenges • Other challenges

  23. Wrap Up • We will regroup • A representative from each group will share with us how his or her group plans to address the challenges associated with their assigned design approach

More Related