Exploratory Factor Analysis

1. Exploratory Factor Analysis

2. Review: FA vs. PCA • Principal components analysis seeks linear combinations that best capture the variation in the original variables. • Factor analysis seeks linear combinations that best capture the correlations among the original variables.

3. Review: FA vs. PCA Principal Components Analysis: The correlation matrix contains ones on the main diagonal, accounting for all of the variance in X—the goal of principal components analysis. Factor Analysis: Seeks to identify common factors that influence correlations among items, it is not a correlation matrix that is analyzed  thus, the main diagonal is replaced by the communalities - the variances in X that are due only to the common factors. But, communalities are not known in advance of the factor analysis, giving rise to the communality problem and the need to solve for the common factors iteratively.

4. FA: Principal Axes Factoring In the principal axes approach to factor analysis, the only difference compared to principal components is that the matrix being analyzed is a correlation matrix (which is also a variance-covariance matrix for standardized variables) in which the main diagonal contains the communalities rather than the variances.

5. Simple Structure • Most of the loadings on any given factor are small and a few loadings are large in absolute value • Most of the loadings for any given variable are small, with ideally only one loading being large in absolute value. • Any pair of factors have dissimilar patterns of loadings.

6. The ideal loadings are the simple structure, which might look like this for the factor loadings: To get closer to the ideal, we rotate the factors

7. Varimax One way to approach this ideal pattern is to find the rotation that maximizes the variance of the loadings in the columns of the factor structure matrix. This approach was suggested by Kaiser and is called varimax rotation.

8. Quartimax A second way to approach this ideal pattern is to find the rotation that maximizes the variance of the loadings in the rows of the factor structure matrix. This approach is called quartimax rotation.

9. Equimax For those who want the best of both worlds, equimax rotation attempts to satisfy both goals. Varimax is the most commonly used and the three rarely produce results that are very discrepant.

10. Example: Principal Components Analysis Data on drug use reported by 1634 students in Los Angeles. Participants rated their use on a 5-point scale: 1 = never tried, 2 = only once, 3 = a few times, 4 = many times, 5 = regularly.

11. Syntax for Factor Analysis FACTOR /VARIABLES cigs beer wine liquor cocaine tranqs drugstr heroin marijuan hashish inhale hallucin amphets /PRINT INITIAL KMO EXTRACTION ROTATION /PLOT EIGEN ROTATION /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION . KMO asks for a test of multicollinearity – this will tell whether there is enough shared variance to warrant a factor analysis PC = Principal Components PAF = Principal Axis Factoring

12. Should the matrix be analyzed? Bartlett's test indicates the correlation matrix is clearly not an identity matrix This can vary between 0 and 1, indicating whether there is sufficient multicollinearity to warrant an analysis. Higher values indicate the desirability of a principal components analysis.

13. Two principal components account for nearly half of the information in the original variables

14. Initial Extraction All principal components are extracted so all of the variance in the variables is accounted for.

15. Part of the loading matrix. How should the first two principal components be interpreted?

16. Only two principal components are indicated by the scree test.

17. Only two principal components are indicated by the Kaiser rule as well.

18. With only two principal components, much less of the variance in each variable is accounted for.

19. The loadings for the first two components do not change when they are the only ones extracted.

20. Common Factor Analysis Drug use data by 1634 students. Participants rated their use on a 5-point scale: 1 = never tried, 2 = only once, 3 = a few times, 4 = many times, 5 = regularly.

21. The analysis begins in the same way as principal components analysis. It would make little sense to search for common factors in an identity matrix:

22. Unlike principal components analysis, factor analysis will not attempt to explain all of the variance in each variable. Only common factor variance is of interest. This creates the need for some initial estimates of communalities.

23. The number of factors to extract is guided by the size of the eigenvalues, as it was in principal components analysis. But, not all of the variance can be accounted for in the variables.

24. Remember, this is because PCA assumes no error of measurement, whereas FA assumes there is measurement error. To the extent there is random error in the measures, the eigenvalues for factor analysis will be smaller than the corresponding eigenvalues in principal components analysis.

25. The location of the factors might be rotated to a position that allows easier interpretation. This will shift the variance, but preserve the total amount accounted for.

26. Two factors appear to be sufficient. The attenuated eigenvalues will generally tell the same story.

27. The loadings will be reduced in FA compared to PCA because not all of the variance in X is due to common factor variance. PCA FA

28. Rotating the factors to simple structure makes the interpretation easier. The first factor appears to be minor “recreational” drug use. The second factor appears to be major “abusive” drug use.

29. New Example: FA A sample of 303 MBA students were asked to evaluate different makes of cars using 16 different adjectives rated on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree): “This car is an exciting car.”

30. Are all 16 individual ratings required to understand product evaluation, or, is there a simpler measurement model?

31. The analysis must begin with some idea of the proportion of variance in each variable that can be attributed to the common factors. The most common initial estimate is the squared multiple correlation between a given measure and all the remaining measures.

32. Three common factors appear to underlie the 16 evaluative ratings.

33. The three common factors account for two-thirds of the common factor variance.

34. Without rotation, the interpretation of the factors is not readily apparent, especially the second and third factors.

35. The varimax rotation makes the interpretation a bit easier. What would you name these factors? Factor 1? Factor 2? Factor 3?

36. It is rare for the different rotational criteria to produce different results

37. PCA vs. FA • Advocates of factor analysis often claim that it is inappropriate to apply principal components procedures in the search for meaning or latent constructs. • But, does it really matter all that much? • To the extent that the communalities for all variables are high the two procedures should give very similar results. • When commonalities are very low  factor analysis results may depart from principal components.

38. PCA FA

39. PCA FA

40. PCA FA

41. FA: Oblique Rotation • Factor analysis offers a more realistic model of measurement than principal components analysis by admitting the presence of random and systematic error. • Another way to make the model more realistic is to relax the restriction that factors be orthogonal. Allowing oblique factors has two potential benefits: • It allows the model to better match the actual data • It allows the possibility of higher order factors

42. FA Example, Oblique Rotation A sample of 303 MBA students were asked to evaluate different makes of cars using 16 different adjectives rated on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree): “This car is an exciting car.”

44. Oblique rotation relaxes the requirement that factors be independent. This requires the addition of a new matrix—the factor pattern matrix. • With orthogonal rotation, because the factors are independent, the weights are simply correlations---just the elements of the factor loading (structure) matrix. • When the factors are not orthogonal, the correlations are not the only weights considered (structure matrix) and a separate matrix containing partial weights is necessary (pattern matrix).

45. Because the factor pattern matrix represents the unique contribution of each factor to the reconstruction of any variable in X, it provides a better basis for judging simple structure. Available techniques for achieving simple structure for oblique rotation (e.g., promax, direct oblimin) confront an additional problem---the specification for the amount of correlation among the factors. Unlike orthogonal rotation in which these correlations are fixed at 0, in oblique rotation, simple structure can be sought for any correlations among the factors.

46. Hypothetical data (N = 500) were created for individuals completing a 12-section test of mental abilities. All variables are in standard form.

47. The correlation matrix is not an identity matrix:

48. The scree test clearly shows the presence of three factors:

49. On average, the three factors extracted can account for about half of the variance in the individual subtests.

50. Factor analysis accounts for less variance than principal components and rotation shifts the variance accounted for by the factors. Why does the sum of the squared rotated loadings not equal the sum of the squared unrotated loadings?