1 / 93

Goals of the Update Briefing

Overview of O*NET Data Collection and Activities OIDAP Meeting Phil M. Lewis David R. Rivkin National Center for O*NET Development Pam Frugoli Employment and Training Administration, DOL May 4, 2011. Goals of the Update Briefing. Overview of O*NET Project Data Collection Program

Antony
Download Presentation

Goals of the Update Briefing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of O*NET Data Collection and ActivitiesOIDAP Meeting Phil M. LewisDavid R. RivkinNational Center for O*NET DevelopmentPam FrugoliEmployment and Training Administration, DOLMay 4, 2011

  2. Goals of the Update Briefing • Overview of O*NET Project • Data Collection Program • Address OIDAP Questions • Products & Tools • O*NET Users • Special Projects

  3. OIDAP Questions • Please describe how the O*NET data collection strategy builds upon the BLS Occupational Employment Statistics (OES) database. • Are you still using Dun & Bradstreet data to find establishments? Were OES data not able to accomplish this, or is it an efficiency issue? • What was the original balance between the Establishment Method and the Occupational Expert Method for populating the O*NET database? Has that changed now that you are in maintenance mode? How are the occupational experts identified? • Can you describe the role of the state workforce development offices in data collection for O*NET? Has this changed over time? • Why did O*NET decide not to use field analysts to collect the data on occupations?

  4. OIDAP Questions (cont.) • What are the major challenges associated with identifying employees in specific occupations to observe in the field? • How do Detailed Work Activities differ from Generalized Work Activities in the O*NET? How were they empirically derived? • We understand that a rule of thumb for statistical sufficiency was set at 15 observations for each domain. Are you still satisfied with that number? • Are you planning any changes in procedure or content as a result of the recent review of O*NET by the National Research Council? • How might your sampling parameters change if the results were subject to legal challenge? • What advice do you have for SSA as they begin to develop an OIS for disability adjudication purposes?

  5. Overview of O*NET Project

  6. Why O*NET? 1980 Work, Jobs, and Occupations: a Critical Review of the Dictionary of Occupational Titles (National Research Council) 1993 Final report of the Advisory Panel on the DOT (APDOT) 1999 The Changing Nature of Work: Implications for Occupational Analysis 2000 O*NET Data Collection Program Survey Pretest 2001 Official OMB approved O*NET data collection using the O*NET survey questionnaires initiated

  7. O*NET Project Team • U.S. Department of Labor, Employment & Training Administration • National Center for O*NET Development • North Carolina Employment Security • RTI; MCNC; HumRRO; NC State University; Maher & Maher

  8. National Center for O*NET Development • Data Collection • Dissemination • Implementation • Research and Development • Technical Assistance/Customer Support

  9. What is O*NET? • A “common language” and dynamic system for describing the world of work for both the public and private sectors • A comprehensive system for collecting and disseminating information on occupational and worker requirements

  10. What is O*NET? • Flexible competency-based system with emphasis on skills transferability • Framework for organizing job and worker information • Data on occupations covering the entire U.S. Economy

  11. What is O*NET? • Uses information technologies to facilitate the collection, storage, and distribution of quality data • A resource for businesses, educators, job seekers, HR professionals, and publicly funded government programs

  12. O*NET Structure • The O*NET-SOC Occupational Taxonomy • The O*NET Content Model

  13. SOC and O*NET-SOC • SOC mandated by US Office of Management and Budget • Developed by multi-agency initiative

  14. Key characteristics of U.S. SOC • Structured for comparability • Unified classification structure • Four hierarchical levels to enable data collectors to choose a level of detail corresponding to their needs and ability to collect data on different occupations

  15. SOC and O*NET-SOC • O*NET-SOC is a SOC based classification that provides a greater level of detail as needed • O*NET-SOC 2010 taxonomy released December 2010 • Currently data collected on 974 O*NET-SOCs • Adds 269 more specific occupations • New and emerging • Different tasks and KSAs

  16. O*NET Content Model

  17. Content Model: Sub-Domains Worker Characteristics Abilities Interests Work Styles Abilities Cognitive Psychomotor Physical Sensory Cognitive Verbal Idea Generation & Reasoning Quantitative Memory Perceptual Spatial Attentiveness Verbal Oral Comprehension Written Comprehension Oral Expression Written Expression http://www.onetcenter.org/content.html

  18. O*NET 15.1 Database The O*NET Database: Version 15.1 • Occupation data • Cross-Occupational + Occupation Specific • 230+ variables • Importance, level, frequency • @ 500 ratings per occupation • @ 3500 metadata per occupation Abilities Scales Reference Content Model Reference Skills Educ, Trng & Exp Categories Survey Booklet Locations Task Categories Educ, Trng & Exp Task Ratings Interests Task Statements Job Zone Reference Supplemental Files Work Activities Job Zones • Related Occupations • Crosswalks • Detailed Work Activities • Emerging Tasks • Lay Titles • Tools and Technology • O*NET-SOC 2000 to O*NET-SOC 2006 • Tasks (Release 5.1 File Layout)‏ • Work Needs • Crosswalks • Detailed Work Activities • Emerging Tasks • In-Demand Occupations • Lay Titles • Related Occupations • Tools and Technology • Work Needs Work Context Categories Knowledge Work Context Level Scale Anchors Work Styles Occupation Data Occupation Level Metadata Work Values

  19. Occupational Level Statistics O*NET-SOC Establishment Response Rate O*NET-SOC Employee Response Rate O*NET-SOC Case Completeness Rate Total Completes for O*NET-SOC Occupational Level Distribution Statistics Data Collection Mode Current Job Tenure Industry The O*NET Database:Metadata • Ratings Level Statistics • Confidence Intervals • Lower and upper 95% bounds • Standard Deviation • Standard Error • Sample Size • Flags • Not Relevant for the Occupation • Recommended Suppression

  20. O*NET Data Availability • 874 occupations • Comprehensive data • 359 second update • 100 occupations • Description, task list, lay titles • Majority also have interests, work values, and tools & technology

  21. O*NET Data Publication Goals • New database released annually • Minimum of 100 occupations updated • Average currency of all occupations = 2.59 years • Priority established by DOL • Maximum 5 years-old • Bright Outlook • Green occupations • Linked to technology, math, and science, computers, engineering, and innovation

  22. O*NET Data Currency

  23. O*NET Data Collection Program

  24. O*NET Data Collection Overview • Proven successful and cost effective methodology designed to collect and yield high quality occupational data • Multi-method approach to provide flexibility within a framework of standardized procedures • Establishment, Occupational Expert, Supplemental Frames, Analyst Ratings, Web-Based • Minimizes public burden and costs • Approved by Office of Management & Budget (OMB)

  25. O*NET Data Collection Overview • Continuous data collection since June 2001 • Three successful OMB Clearances • Comprehensive update by job incumbents and occupational experts of the 2006 O*NET-SOC Taxonomy • Transition to 2010 O*NET-SOC • New & Emerging occupations • Unparalleled partnership between Department of Labor and private/public community • 40,000+ business/organizations • 160,000+ job incumbents/experts • 450+ National Associations

  26. Data Collection Overview (cont.) • High quality data from a national sample of job incumbents/occupational experts • Strong business participation • 76% plus response rate • Strong employee participation • 65% plus response rate • Strong occupation expert participation • 82% response

  27. Data Collection Overview (cont.) • Web-based case management system used to control all sampling and data collection systems • Case management, BL contacts with establishments, questionnaire and informational mailings, questionnaire processing, inventory control, etc. • Finely tuned procedures, systems and infrastructure capable of surveying multiple occupations simultaneously • Capability developed, tested, and enhanced over 11 years

  28. Sources of Occupational Data • Job Incumbents and Occupation Experts • Education, Job Titles, Knowledge, Tasks, Work Activities, Work Context, Work Experience, Work Styles • Occupation Analysts • Abilities, Skills • Web-based Research • Detailed Work Activities, Green, Tasks, Tools and Technologies (T2)

  29. Establishment Method • Two stage sample • Business establishments - POC • Job incumbents within business establishments

  30. Establishment Method (cont.) • Job incumbents complete one of three survey questionnaires (25 -30 minutes) • Generalized Work Activities, Knowledge/Work Styles, or Work Context • Task List • Background Info • Incumbents choose response option • Paper-and-pencil • Web-based (approximately 25%)

  31. Design of Collection Waves • Identify ~50 primary occupations to target in a sample wave • Wave X.1: Designed to get 34% of sample • Wave X.2: Designed to get 33% of sample • Wave X.3: Designed to get 33% of sample • Wave X.4: Sample residual

  32. Design of Collection Waves (cont.) • Each wave is a cluster of similar occupations • Secondary occupations which are found across industries are also included to maximize efficiency • Multiple sub-waves allow for greater precision • Locating occupations • Controlling public burden and project resources

  33. Stage One Sampling • OES data from BLS used to determine the initial industry distribution for each occupation • Sample business establishments selected from database of business locations

  34. Stage One Sampling (cont.) • OES data from BLS used to determine the initial industry distribution for each occupation • Indicates which industries occupations are employed in and the share and distribution of occupational employment across industries • Does not contain information on establishments

  35. Stage One Sampling (cont.) • Industry information for each occupation is refined by O*NET Center analysts • Review and face validity checks • For example, religious institution sub-section removed from service industry if sampling for bartenders • Determine industries to include based on overall distribution and population coverage goals • Refined/target by experience from previous updates, when available

  36. Stage One Sampling (cont.) • Sample business establishments selected from a frame of business locations • Dunn & Bradstreet (D&B) database • ~15 Million establishments • Info obtained from multiple sources • Tax records, credit reports, telephone directories • Updated continuously on a monthly basis • Links to SIC and NAICS industry information

  37. Population Coverage • Gather data on the “core” of the occupation • Where the majority of incumbents employed • Average coverage level is 85%

  38. Population Coverage (cont.)

  39. Stage Two Sampling • Led by highly qualified O*NET Business Liaisons (O*NET BL) • Full time staff working in dedicated call center • Educational and work experience criteria higher than typical telephone interviewer • The sampled establishment’s Point-of-Contact (POC) works with the O*NET BL to identify the a list of eligible employees • Identification Profiles (ID Profile) are used when asking POC if occupations are present • Helps ensure accuracy in matching employees to occupations

  40. Stage Two Sampling (cont.) • Automated, random selection of job incumbents from the POC’s eligible list • Limits placed on POC burden • No more than 5 occupations sampled • Never more than 20 employees selected • Can only be included within the data collection once each year • Selected employees • Asked to complete the questionnaire on their own time • Responses remain anonymous and confidential from both the employer and the public • Complete via the web or mail back directly • All individual identifiers removed

  41. Model-Aided Sampling (MAS) • Innovative sampling approach that reduces data collection cost and burden to the public by preventing occupations from greatly exceeding their targeted sample • Builds on existing sampling paradigms: traditional and model based • For each occupation, a targeted sample size for specific demographic domains is modeled • Census region • Establishment size • Industry division • Data collection is halted in a MAS cell when the targeted respondent sample size is projected to be achieved

  42. Summary of Establishment Method Sample Selection Process 42

  43. Establishment Method Data Collection Protocol 43

  44. Other Key Features • Incentives • Employer • Toolkit for Business • POC • Clock, Certificate of Appreciation • Employee • $10 • Outreach to professional/trade associations increase awareness • Endorsement list provided to POC

  45. Supplemental Frame Method • Multiple strategies to augment the establishment data collection • Supplemental Frame Incumbent • Job incumbents are directly accessed via an association listing to complete the data collection • e.g. Industrial Organizational Psychologists • Supplemental Frame Establishment • Targeted employer sample developed via expert contact/associations where the sampling frame coverage is significantly high but is not adequate by itself • e.g. Freight and Cargo Inspectors • Special Frame Establishment • Targeted employer sample completely developed via expert contact/associations where coverage is extremely high • Normal establishment method is bypassed • e.g. Nuclear Power Reactor Operators; Flight Attendants

  46. Occupational Experts (OE) Method • Used when occupation is difficult to locate in establishments • Small employment size • Job incumbents inaccessible due to work in remote locations • New and emerging occupations

  47. OE Method (cont.) 47 • Data collected from experts in target occupation • Supervisors, trainers, others with extensive knowledge of occupation • Identify appropriate source organizations (e.g. professional associations) • Good coverage of occupation • Can identify members who are occupation experts • Willing to provide lists of experts

  48. OE Method (cont.) 48 Select sample from membership lists Contact, screen, and survey OEs directly – no establishment or POC OEs complete all three domain questionnaires, background and task questionnaires OE incentives – clock, Certificate of Appreciation, $40

  49. OE Method DataCollection Protocol 49

  50. Analyst Ratings Method • Occupational Analysts Rate the Ability and Skill Domains • Updated occupation information collected from job incumbents used to describe occupation and assist with the rating process • Extensive training and quality assurance procedures

More Related