1 / 13

An example of a CFA: Evaluating an existing scale for Work Locus of Control

An example of a CFA: Evaluating an existing scale for Work Locus of Control. Thought for the day:

gaille
Download Presentation

An example of a CFA: Evaluating an existing scale for Work Locus of Control

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An example of a CFA:Evaluating an existingscale for Work Locus of Control Thought for the day: An airport screener lost her job at the Newark Intl. Airport because she failed to find a butcher knife in the purse of a passenger. The passenger, Katrina Bell, was not charged with a crime because she confessed that she had forgotten that she had placed the knife in her handbag the previous week as she headed out for a “blind date”.

  2. Normal operating procedures • Most people who do research, search for existing measures and if they find one that measures one of their constructs, then they just use it. • I would encourage you to be more critical. In particular, did the author of the scale do the proper groundwork? • I will take you through an example which I hope will convince you that one benefits from being skeptical and careful.

  3. Evaluation of the Spector Work Locus of Control Scale • This scale measures attitudes about how much control a worker has in his/her work situation. There is a large literature on “internal and external loci of control”, and this measure is a specific example of this type of scale. • Spector wrote the scale so that half of the items are “external” items: • “promotion is usually a matter of good fortune”, • And half are “internal” items: • “promotions are given to employees who perform well on the job”. • What happens if we do a Cronbach’s alpha on the entire scale (16 items)?

  4. Results • All 16 items: a = .597. • One strives to obtain an alpha of at least .70, so this is subpar. However, does this make sense? • Spector felt that there were two discriminable factors, so we should really do the internal reliability analyses on the two factors separately: • Internal (8 items): a = .828; and • External (8 items): a = .813. • That’s what we would expect.

  5. One factor,or more? • Spector originally designed the scale so that the external items are reverse-scored and then added to the internal items. What does this mean? It suggests that he thinks that there is a single factor operating here. • A simple test can determine whether these two clumps are really a single factor: a correlation. If the correlation is highly significant AND negative, then you would have evidence for this point of view. • What did we find? I obtained: Pearson r(145) = -.360, p < .001. It is negative, but not “highly significant”. This should cast some doubt on his suggestion.

  6. Psychometric work • And let me point out that he didn’t perform even an exploratory factor analysis on the scale when he first published a research paper on it (Spector, 1988). • Were psychometric techniques like factor analysis available in 1988? Yes, they were. Bear this in mind.

  7. If you do an EFA, what do you find? • We did a PCA with varimax rotation. • Expected to find that two factors was optimal: what did we find?

  8. Table 2. Rotated Pattern matrix of Work Locus of Control Scale PCA. Item Factor 1 (External) Factor 2 (Internal 1) Factor 3 (Internal 2) 1. Promotion is usually a matter of good fortune. .81 -.03 -.08 2. Making money is primarily a matter of good fortune. .77 -.13 -.14 3. The main difference between people who make a lot of money and people who make a little money is luck. .75 .02 .06 4. Getting the job you want is mostly a matter of luck. .72 -.06 .05 5. In order to get a really good job you need to have family members or friends in really high places. .72 .05 .17 6. To make a lot of money you have to know the right people. .71 .02 -.14 7. When it comes to landing a really good job, who you know is more important than what you know. .67 .01 -.16 8. It takes a lot of luck to be an outstanding employee on most jobs. .67 -.08 -.02 9. Most people are capable of doing their jobs well if they make the effort. -.03 .81 .08 10. A job is what you make of it. .03 .71 .32 11. On most jobs, people can pretty much accomplish whatever they set out to achieve. .14 .68 .07 12. If you know what you want out of a job, you can find a job that gives it to you. -.19 .57 .33 13. If employees are unhappy with a decision made by their boss, they should do something about it. -.10 .35 -.17 14. People who perform their jobs well generally get rewarded for it. -.11 .13 .82 15. Promotions are given to employees who perform well on the job. -.08 .22 .73 16. Most employees have more influence on their supervisors than they think. .03 -.01 .65

  9. Then we did a CFA in LISREL • We examined three models: • One factor; • Two factors (Int and Ext); and • Three factors (from the previous page). • A CFA gives you a number of “model fit indices” to tell you how well the data fit the proposed model. • How did they turn out?

  10. Table 5. Fit indices for the WLCS* factor models for Study 2 Fit Indices Model 2 df 2 / df GFI NNFI RMR RMSEA One factor model 814.3 104 7.83 .74 .73 .12 .155 Two factor model 344.9 103 3.35 .87 .89 .07 .091 Three factor model 253.4 101 2.51 .90 .92 .06 .073 *WLCS; Work Locus of Control Scale, GFI; Goodness-of-Fit Index, NNFI; Non-Normed Fit Index, RMR; Root Mean Square Residual, RMSEA; Root Mean Square Error of Approximation Model fit indices

  11. Three factor model looks best • The model fit indices are superior (and mostly acceptable) for the 3-factor solution. • This is strong evidence to say that the scale is psychometrically flawed. • Why would one factor (the Internal factor) fragment and fall apart? Heterogeneity of item meanings would be my guess. • Joe Oliver and I just succeeded in getting a paper published which reports these difficulties. Was rejected initially by Spector or a crony.

  12. How does one derive a good scale? • Begin with qualitative data—ask workers about ways in which they feel in control or not. • Write a whole bunch of questions (I would start with about 40). • Do a pretest with a large sample (300-400?), and select items that are highly intercorrelated with each other in these two domains. • Do another data collection with about 24 items (12 each), and select the top 8 or 10 in each domain. Verify that they clump into two discriminable factors with EFA. • Obtain another large dataset, and perform a CFA. If you’ve done your work well, the CFA will confirm that you have a good factor structure. • Now publish. • Sit back and wait for the royalty checks to roll in.

  13. Moral of the story • Almost no one does what was described on the previous page (except people who publish their tests to make money). • Most people write some questions, collect some data, do an EFA, and publish those results. • Why are we surprised when we find that the measure doesn’t work psychometrically? • If you create a scale, do the groundwork.

More Related