1 / 88

What’s Up DOT?

What’s Up DOT?. IARP Occupational Database Committee IARP Forensic Conference Hyatt Regency Bonaventure Weston, Florida October 31, 2008. Scope of Project. IODC → formed spring 2007 Purpose: Identify replacement of DOT Focus: research past efforts to update DOT

Albert_Lan
Download Presentation

What’s Up DOT?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What’s Up DOT? IARP Occupational Database Committee IARP Forensic Conference Hyatt Regency Bonaventure Weston, Florida October 31, 2008

  2. Scope of Project • IODC → formed spring 2007 • Purpose: • Identify replacement of DOT • Focus: • research past efforts to update DOT • current uses of DOT & O*NET • identify & evaluate other occupational databases • develop support to update/replace DOT

  3. Scope of Project • Identified two occupational databases: eDOT, a product of Economic Research Institute; McDOT, a product of Vocationology, Inc. • Third database, based on the Common Metric Questionnaire (CMQ) may be ready in few years • Arranged to have training and use of these databases for study purposes. • First, needed to learn more about DOT

  4. History of the DOT • Following Civil War, U.S. and Census Bureau began to classify occupations • First occupational dictionary: 1927, A Dictionary of Occupational Terms, Great Britain Ministry of Labour • Wagner Peyser Act, 1933: United States Employment Service (USES)-- match workers with jobs.

  5. History of the DOT • Work on the DOT began around the same time; 1st Edition 1939 • Subsequent editions published 1949, 1965, 1977 • Supplements came out 1982, ’86, ’91 • 20 new occupations added in 1998 • Intended purpose: to assist USES in placing workers in jobs. • 1st Edition: skilled, semi-skilled or unskilled

  6. History of the DOT • 1965 edition: worker traits, worker functions (Data, People, Things), Occupational Group Arrangements • 1977 edition: ~75,000 job analyses; 2,100+ new occupations; eliminated gender bias in job titles and descriptions; included detailed worker characteristics (SCO) • 1986 ed. added 761 new occupations • 1991 added 844, deleted 208; added GOE codes, GED, SVP, DLU

  7. History of the DOT • Each definition includes: code, title, industry designation, alternate titles, body of definition, undefined related titles, and definition trailer (GOE, strength, GED-RML, SVP, DLU) • Basic concepts described in the DOT occupational definitions: element, task, position, job, and occupation.

  8. History of the DOT • Major elements of an occupation: • What the worker does (data, people, things) • What gets done (work fields-WF) • Skill level to perform this work (SVP) • End product (materials, products, subject matter and services-MPSMS)

  9. History of the DOT • DOT → 4 classification systems: • by job content (OGA) • by worker function (DPT) • by industry affiliation (Industry Designation) • alphabetically by title

  10. History of the DOT • Job analysis → basis of the DOT • National Research Council (1999) defines occupational [job] analysis: "the tools and methods used to describe and label work, positions, jobs and occupations" • DOT defines an occupation as: “a collective description of individual jobs performed, with minor variations, in many establishments”

  11. History of the DOT • The Handbook for Analyzing Jobs → job analysis methodology used in creation of DOT. • HAJ first published in 1944; 4th Edition in 1991 (RHAJ). • 1972 Edition published halfway through completion of 1977 DOT. • Two primary formats in job analysis: work-oriented—what gets done; and worker-oriented—what the worker does. HAJ/RHAJ uses both formats

  12. History of the DOT • The Selected Characteristics of Occupations Defined in the Revised Dictionary of Occupational Titles (1993) • 1st edition 1966, updates in 1968, 1981, and 1993. • Includes DOT titles arranged by: GOE; SVP; strength; physical demands; environmental conditions; index of titles with DOT codes; definitions of the worker traits and functions

  13. History of the DOT • Positive factors of the DOT: • use of skilled job analysts • task level descriptions • foundation built upon data people things • Worker traits, characteristics • Useful for TSA: MPSMS, WF, SVP • Attempt to cover national economy • Depth of information

  14. Development of the DOT • Sample for DOT: all jobs in the US economy • County Business Patterns/Thomas Registry → used to identify # establishments in each industry. • Industries assigned to one of 11 field centers → average 42 industries/field center • “any industry” designation given to one field office

  15. Development of the DOT • Each analyst identified which establishments to contact • Attempt to include one small, one medium and one large size establishment for each industry, and to focus on new and emerging occupations. • No clear supervision during process. • Appeared primary criteria for selecting establishment → proximity to field office • Employers: right of refusal; no incentives

  16. Development of the DOT • Employers controlled which jobs were analyzed, and which employees were chosen for analysis. • Analyst chose which jobs to observe. • If job analyzed for the 3rd Edition, may complete abbreviated analysis or none at all. • If another analyst had recently completed a similar analysis, none would be done. • Observe 1-2 workers per job.

  17. Development of the DOT • Variety methods to record data: HAJ format; abbreviated format; or simple notes. • Analysts not allowed to bring in tools or equipment to measure certain aspects of jobs—estimation and observation only. • Methods for rating worker traits were vague—particularly for aptitudes, temperaments and interests.

  18. Updates of the DOT • Much time has passed since any onsite job analyses of DOT occupations

  19. How the DOT is used • Basis for transferable skills analysis • Primary uses: career & voc counseling, SSD • Secondary uses: library reference, personnel management, employee placement, govt. uses, research, others

  20. O*NET • Outgrowth of the Advisory Panel on the DOT (APDOT) from the early 1990s • Dept. of Labor “replacement” for the DOT • Skills-based database rather than task-based dictionary • Preliminary version released Dec. 1997 with first version (O*NET 98) out Dec. 1998

  21. O*NET • Utilized SOC coding rather than DOT • Current version is O*NET 13 • The 12, 761 DOT occupations have been aggregated/collapsed to 812 groups • Composite information from many jobs; not intended to describe a particular job • Uses mean data rather than mode used by DOT

  22. Position Analysis Questionnaire (PAQ) • PAQ’s business: create useful work and labor economic reports • Structured job analysis; 187 items (plus 8 items re: comp.) called “job elements” • Job elements are worker-oriented;6 domains: information input; work output (physical activities and tools); mental processes; relationships with others; job context (physical and social environment); and other work characteristics (such as pace and structure)

  23. PAQ • PAQ dedicated solely to job analysis services since 1972 • Terminology, definitions and rating scales different than that used in DOT • Requires post-college graduate reading level • Job analysts or supervisors usually complete PAQ

  24. PAQ • PAQ trains subscriber employers in job analysis techniques—in person and online • Limited coverage of managerial, supervisory, executive and professional work led to development of separate questionnaire • Average reliabilities—item and re-rate—range .68-.80

  25. PAQ/DAQ • DAQ: Disability Analysis Questionnaire. Developed by PAQ • Includes 99 questions from the PAQ that are most closely related to items from the DOT/HAJ • ERI modified DAQ slightly—made certain the questions best-matched those of the DOT • The DAQ questions are the ones asked of incumbents when visiting websites

  26. eDOT • The “enhanced DOT” -- database and software program developed by Economic Research Institute (ERI) • ERI began as compensation information provider in 1987 • Started the eDOT Skills Project in 2002 to collect data and update the DOT • In 2004, ERI purchased PAQ; PAQ operates eDOT Skills Project under a license with ERI • Database has two parts: archived DOT and new eDOT which includes the old DOT

  27. eDOT • Includes 64 SCO characteristics + 35 new factors such as keyboarding, education, mental and cognitive factors, etc. • Includes 20 measures from SSA’s Mental Residual Capacity paper • Various people contribute data: subject matter experts (job analysts trained in the use of the PAQ; voc rehab counselors; major disability carriers; workers compensation analysts; and primarily incumbents visiting websites Career Builder, SalaryExpert, SalariesReview.

  28. eDOT • Each data point has associated reliability, standard error and deviation calculated • 1,000-1,500 job analyses done per year, including those by incumbents • Over 1 million PAQ job analyses included in eDOT (completed since 1974) • Sample is one of convenience • Control for sampling error by using multiple sources of data

  29. eDOT • Differences between eDOT and DOT • Different rating scales, definitions, terms • eDOT→ uses revised version of SIC which it developed, called eSIC • eDOT→fluid database; DOT→fixed • eDOT→convenience sample; DOT→attempt to capture all jobs in the national economy • Electronic v. paper • PAQ→ interviewing worker, not always observing; DOT→observation

  30. eDOT • Examples of new jobs added to eDOT computer sys admin 030.162-500 call center rep 299.357-201 asst. mgr retail store 185.167-505 executive asst 169.167-911 sales assistant 209.562-800 maint. helper 806.687-011 CAD/CAM Tech 017.262-519

  31. eDOT • ERI/PAQ “masks” jobs • Criteria for exclusion: • job analyses over 15 years old • face validity (abalone diver) • 10 or less requests for the job • not listed on job boards or salary surveys • not mentioned on any loan applications • not on any labor/proxy/form 990 • not mentioned on Calif. state work comp

  32. eDOT • Examples of “masked” jobs in eDOT: animal breeder410.161-010 military recruiter 166.267-026 pres., financial inst. 186.117-054 police commissioner 188.117-118 feed blender 520.685-094 collator operator 208.685-010

  33. eDOT • As of January 2008, ERI had • Added 717 new occupations • Removed 4,103 occupations (no/low frequency) • Verified the existence of 8,658 occupations • Total of 9,375 occupations in eDOT, compared to 12,761 in the DOT and 812 in O*NET • As of July 2008, have added WF and MPSMS to all jobs in eDOT

  34. McDOT History • McCroskey Dictionary of Occupational Titles (McDOT) • Part of the McCroskey Vocational Quotient System (MVQS) • Based on VDARE • McDOT includes both the old DOT and O*NET. • Data has been fused from both sources, identifying 24 most significant worker traits

  35. UNUM: Project with eDOT • Methodology for selecting eDOT • Findings • Future Directions

  36. Issues with DOTMiller et al. Study 1980 • Work, Jobs and Occupations: A Critical Review of the Dictionary of Occupational Titles • Comprehensive review & evaluation of the 1977, 4th Ed. DOT, conducted on behalf of the National Research Council, at the request of the Department of Labor

  37. Issues with DOTMiller et al. Study 1980 • “The comprehensiveness, reliability and accuracy of the DOT are in large part a function of the data collection and analysis procedures used to produce it.” p.114

  38. Issues with DOTMiller et al. Study 1980 • Data Collection/Methodology Issues: • Lack of written procedures on how DOT produce • Majority of principles used to create DOT were established in 1939 & 1949 • Sampling plan “complicated and indirect,” did not include all jobs in US economy

  39. Issues with DOTMiller et al. Study 1980 • Data Collection/Methodology Issues: • Heavy orientation to manufacturing sector • Limited review of jobs requiring cognitive skills over physical skills • Inadequate update schedule

  40. Issues with DOTMiller et al. Study 1980 • Found analysts often unable to provide concrete explanation for how they rated worker traits besides subjective means or past experience • Some occupations not reviewed at all; others reviewed excessively: material handler → 652 job analyses • 1979 study random job titles found 81% of 4th Ed. descriptions exactly the same as the 3rd Edition

  41. Issues with DOTMiller et al. Study 1980 • Data Collection/Methodology Issues: • 16% occupational descriptions completed without a single job analysis • 29% based on one job analysis • 19% based on two job analyses • 36% based on three or more analyses

  42. Issues with DOTMiller et al. Study 1980 • Reliability/Validity Issues: • Validity: measurements of worker traits/functions found to be fairly unreliable: “vague and ambiguously defined. Not readily apparent what the variables are intended to measure” (p. 164) • Worker traits/functions developed in the 1950s—questionable validity for today’s labor market • Question whether GED and SVP measure prestige or social status of occupations

  43. Issues with DOTMiller et al. Study 1980 • Reliability/Validity Issues: • GED scale validated against curriculum content, not validated for occupational performance • Working conditions & physical demands appear to be developed for unskilled, physical jobs • Reliability: items are scored subjectively; raters had trouble assigning some factors, particularly SVP and aptitudes • Reliabilities higher for manufacturing jobs than for service jobs

  44. Issues with DOT • Cain et al. (1983) study on reliabilities of different ratings. • Two job descriptions per 24 occupations; experienced analysts rated the factors • Found acceptable reliabilities for: data, people, GED reasoning, GED language and SVP • Modest reliabilities for things, GED math, strength factors and location • Reliabilities higher for manufacturing jobs

  45. Issues with DOT • Botterbusch (1993): DOT weaknesses: • Data people things not actually a hierarchy—data is, people is not and things is two hierarchies • GED not directly related to education • SVP does not distinguish between formal and informal training • Difficulty defining, using and defending temperaments • Too many titles

  46. Issues with DOT • Dunn & Growick (2000): weaknesses: • Failure to include variables such as org. setting and worker responsibility level • Low reliability of worker traits • Redundant classification system • Skills that are psychometrically questionable, such as GED and SVP • Ambiguous definitions of worker traits • Questionable validity of worker traits

  47. Issues with DOT • Harvey (Fine et al. 2004) addressed weaknesses of DOT: • Legal and psychometric defensibility in the use of holistic scales to rate worker traits—tend to show low inter-rater reliability and low discriminant validity • Lack of reasonable update schedule

  48. Issues with DOT • National Research Council (1999) • “Unwieldy size” • Growing differences between job descriptions and jobs as they actually occur in the labor market today • Too much detail in each definition • Jobs described by task

  49. Issues with DOT • National Research Council cont. (1999) • Lack of information on factors such as skills, abilities, and knowledge requirements that are either not collected or not reported • Little or no reported information on cognitive factors • Expense of updating • Difficulty of linking it with other databases

More Related