Developing Job-Related Test Materials Using the CATS SYSTEM C ODESP A utomated T est S ystem

1. Developing Job-Related Test Materials Using the CATS SYSTEM CODESP Automated Test System

2. WHAT IS CODESP?Cooperative Organization for the Development of Employee Selection Procedures • A nonprofit consortium of over 700* public school districts, community colleges and charter schools which was formed in 1973 to combine resources to develop classified test materials • Products include: Written Test Items, Interview Questions, Supplemental Application Forms, Performance Exams, Sample Tests (including Instructional Aide) and Writing Exercises * Includes over 300 small districts who have free membership through their County Office of Education because their ADA is less than 1,000.

3. How Can CODESP Help? • CODESP’S Item Bank, soon to be called (CATS), contains over 17,000 items covering most job classes found in public school districts • A job-related test can be easily developed from the items CODESP provides • Multiple types of assessment tests are available

4. How Can You Help? You can be instrumental in creating job-related exams By Conducting SME item reviews and Job Analysis And Notifying CODESP regarding SME Review and Job Analysis results to update test items

5. Employment-related lawsuits may occur: a single legal challenge by an aggrieved applicant can cost thousands of dollars in government fines and tort liability Wasted recruitment time & money when positions must be reopened Wasted salary and benefits to employees who are poor matches or who only stay a limited time Wasted training dollars and time Poor service to customers Reduced morale among current employees who must work with non-qualified co-workers If Insufficient Attention is Given to the District’s Employee Selection Process:

6. Ideal Selection Results In Job-Related/Valid Selection Devices/Tests Accurately Measured Candidate Ability Increased Quality of Candidates Selected Successful Organizations

7. Content Validity • Referred to in the Uniform Guidelines and many court cases: • The content of the test must represent important job-related behaviors • Test items should be relevant to and measure directly important requirements and qualifications for the job (KSAs) • Also known as “job-relatedness” or face validity • Achieved through job analysis

8. Job-Relatedness • To achieve content validity each test item must be related to the job • If not, the validity of the selection procedure is jeopardized • Validity  legally defensible

9. Court Case Griggs v. Duke Power Outlined the need for conducting an analysis of the job for which a selection procedure has been developed • Artificial barriers which are not job-related should be removed from the selection process • Tests should assess minimum requirements needed at the time of hire • Tests should measure how the person will potentially perform on the job, not the person in the abstract

10. The Problem… • It is difficult for HR professionals to be “experts” and make judgments about the job relatedness of test items in all job subject areas

11. The Solution… • Gather information about the job and test items from “SMEs” Subject Matter Experts

12. How Do We Find SMEs? • Use existing knowledge of district personnel • Contact schools/department heads for suggestions • Build lasting, positive partnerships with departments and SMEs for future projects • If you are unable to find SMEs, contact CODESP, or other member districts directly

13. Steps To Take When Working With SMEs To Develop A Selection Test: • Conduct a job analysis or check to see if major changes have occurred since the last job analysis (w/SME) • Review current Job Description 3) Obtain potential test items (create them yourself or obtain them from CODESP) 4) Review potential test items (w/SME) • Select the best items for the test • Create and Print test online using CATS

14. Purpose Of Job Analysis • Ensures that your current job description is correct and up to date • Identify the knowledge, skills, and abilities (KSAs) required at the time of hire to perform the job successfully • Is required to increase the validity of the test items

15. Job Analysis Process • CODESP’s new Job Analysis Tools: • Job Analysis Questionnaire • Job Description Builder • Products will be available Summer ’07 • See Job Analysis Basics Training under Resources / Training Presentations • See Resources / Job Description Builder and Questionnaire

16. Item and Job-Level Match • It is important to determine whether the knowledge measured by an item is essential for the successful job performance of: • entry level employees • intermediate level employees • high level employees • Items should only test knowledge that is essential at the time of hire

17. Item Writing Guide The following are a series of hints and suggestions to help your Subject Matter Experts construct or review test questions: • Write out the full title when describing objects and procedures. Only use acronyms when you are positive that your audience knows exactly what you are describing. When in doubt, spell it out. • Clearly indicate the correct answer for each question. There should only be one “best” answer.

18. Stay away from slang because these words can be open to different interpretations • Try to avoid distracters that include words like “always” and “never” because they are generally false statements as well as words like “some” and “usually” because they are generally true statements • Ask questions that have definite answers so that most experts will agree on the correct response • Try to have a balance of easy, moderate, and hard questions. All disciplines need to be balanced with respect to difficultly level. • Don’t ask for details that only individuals who have worked in that particular department would know. • Watch for hints in the stems (e.g., synonyms). • Do not create questions that have negative stems (“Which one of the following is not…”), unless you are addressing a certain action that all employees need to should be never done. • Questions that have complex answers are less efficient and provide less reliable results.

19. PROBLEM ITEMS • Not applicable to job/field • Terminology too complex • Outdated • Incorrectly keyed • Question Unclear • Choice Unclear • Other choices are also correct • Other choices are too obvious • Choices are not parallel

20. SME Test Item Security • Use A SME Background/Security information form (found under Resources / Exam Planning) and keep it in the test file • Make sure that CODESP test materials are stored in a secure location

21. CODESPCATS SYSTEM CODESPAutomatedTestSystem

22. Getting Started To get started, launch an Internet Explorer session and enter the following web address: http://www.codesp.com. Then, press the Enter key on your keyboard. • This section will cover the following topics: • Website Navigation • Registering for Services • Logging In • Logging Out

24. Navigating the CODESP Website • The CODESP banner and the navigation menu make information readily accessible, making website navigation a breeze. • Some information is only accessible to registered members who have logged in.

25. Helpful Hints • Because this website makes use of pop-ups, it is important to disable any pop-up blocker software that may be installed on your machine. You may be able to disable pop-ups on our website without disabling pop-up blocker software entirely depending on the software installed. • Graphical icons are often used to simplify website interface. Simply, hover your mouse over the icon to view the functionality.

26. you Member Name Navigating the Website Your member start page provides you with a customized view of CODESP training events, your class registrations, newsletters, announcements and your test material request status.

27. Currently Registered Members • The next several slides provide instructions on registering for CODESP • If you are already registered, skip to Slide 55

28. Registering for Services • Becoming a CODESP member is easy and offers many benefits. With CATs CODESP members have access to: • Test Materials • Tutorials • Job Analysis Questionnaire • Job Description Builder • Training • HR Resources • Learn more about the different memberships CODESP offers by clicking on the Membership tab found on the toolbar.

29. Registering for Services Continued • To become a member of CODESP, click on Register Now and complete the following requirements: • Application for Services • Security Agreement • Membership Agreement *   • Purchase order or check for the full amount of the membership fee * • *Not needed by districts and Charter Schools with an ADA of 1,000 or less if County Office is a member

30. New Members Register on the CODESP Home Page Once you are registered you will receive a Member Log-in Name and Password so that you can access the secure areas of the website.

31. To begin the registration process, select the desired membership. Mid-Year is only available from Jan – June and requires a commitment for a full-year membership the following year. * Districts and Charter Schools with an ADA less than 1,000 do not need to complete these requirements, as long as their County Office is a CODESP member. They only need to complete an application and security agreement. Their membership is free.

32. The CODESP Registration Wizard guides you through each step of the registration process: • Step 1: Application for Services • Select your County and District from drop down menu. • Verify the address. • If address is incorrect, make the appropriate changes. • Complete the rest of the form, including a primary and secondary contact person(s). • Click Next to move to the following screen.

34. If your district does not appear on the drop down menu, click What if my district is not listed above? • Fill out the form as seen in the next figure and click the submit button. You will be contacted by a CODESP representative in order to verify your request. Once CODESP has added your district, you will be able to complete the registration process.

36. Step 2: Membership Agreement • Read the Membership Agreement thoroughly. • To comply, click “I agree with the terms and conditions as stated above.” • Click Next to move to the following screen. • Note: You will not be able to continue registering unless you accept the terms.

38. Step 3: Security Agreement • Similarly to Step 2, read the Security Agreement thoroughly. • To comply, click “I agree with the terms and conditions as stated above.” • Click Next to move to the following screen. • Note: You will not be able to continue registering unless you accept the terms.

40. Step 4: Registration Information Review • Verify that the registration information provided is correct. • Click Edit to revise the information. • Click Next to move to complete the CODESP Registration Wizard.

42. Step 5: Print, Sign and Fax Agreement • CODESP is legally obligated to keep a signed contract on file for every CODESP member. • Please print all forms. • Sign them. • Fax or mail them to the CODESP office: • 20422 Beach Blvd. Suite 310 • Huntington Beach, CA 92648 • Fax: (714) 374-8225

44. The Registration Wizard ends at Step 5, but remember, in addition to sending the signed agreements, don’t forget to send your membership payment. Checks and purchase orders are accepted. Forward an Invoice to your Business Office. An invoice is available in the Document Library. Once CODESP has received the membership payment in full and the signed agreements, CODESP will notify the primary contact and your district will be given a District Administrator account. The District Administrator will be assigned a login name and password. To learn more about the District Administrator role and responsibilities, see section titled, District Administrator Account.

45. Logging In If your account has already been established, click Log In. Otherwise, contact your District Administrator in order to set up an account.

46. Logging In Continued First time users must agree to the terms of the security agreement even if one has been completed by other district users:

47. Logging Out Throughout the course of the day, there will be times in which you will not need access to the system—lunch time, breaks, meetings, etc. In these instances, you will need to exit the system. In order to maintain the highest level of security, end your session by clicking on Log Out. To Keep the Site Secure – Always Log-Out When You are Not at Your Workstation

48. My Profile Your profile can be viewed by clicking My Profile, located on the navigation bar. You will notice certain fields are editable, while others are for viewing purposes only. For security reasons, a person cannot change their own Status or Security Role. This data can only be changed by another District Administrator. Never use another person’s log-in name and password

50. Membership • District Directory • Your district will benefit from networking with other districts. The District Directory serves as a CODESP phone book and lists all the districts that are CODESP members. • To access the District Directory, click District Directory located on the navigation bar. • The searching tools provided make it easy to find a district’s contact information. You can search by text or by county. • Searching by Text • Click Text as it is highlighted in the following figure. • Enter keyword(s) in the Search For field. • Click drop down menu to view the searchable fields. • Highlight the field you wish to perform a search on. • Click Search Now