Reliable and valid survey design for structural equation modelling

1. Reliable and valid survey design for structural equation modelling Guanglun (Michael) Mu Center for Learning Innovation, QUT Faculty of Education Supervisory team: Dr. Karen Dooley, Paul Shield & Dr. Catherine Doherty 16 February, 2011

2. RQ1. Does Chinese Australians’ “Chineseness” have positive impacts on their CHL proficiency through the investment of their various resources? RQ2. Does Chinese Australians’ CHL proficiency positively influence their “Chineseness” and further, influence their embodied cultural capital in the CHL field? Research questions

3. Paradigm / Worldview: Post-positivism Theoretical lens: Structuralism Methodological approach: Quantitative research Research method: Survey design Population Instrument Sampling Analysis Overview of research design

4. Population: Chinese Australian urban young adults Target population: Chinese Australian urban young adults, living in Sydney, Melbourne, and Brisbane, ranging in age from 18 to 30 Sample • Why “urban”? • Why “young adults”? • Why “Sydney, Melbourne, and Brisbane”? • Why “18-30”?

5. Instrument development: Questionnaire design • Online questionnaire Key Survey • Measurement scale Nominal, ordinal, interval & ratio 7-point Likert scale as interval scale • Operationalization Validity: Accuracy Face validity & Content validity Reliability: Consistency Test-retest reliability & Inter-item reliability • Strategies

6. Theoretical concepts as latent variables Operationalization Construct items as indicators • ‘Latent variable’ here refers to a variable that is not directly measurable but related to several variables that can be measured directly (Field, 2005). • These directly measurable variables are called ‘indicators’ or ‘measurements’ and can be operationalized through corresponding items in a questionnaire (Field, 2005).

7. Questionnaire design strategies • Use filtering techniques • Offer enough variance • Understand whose opinions are under scrutiny • Avoid double barreled questions • Restate negatively worded questions • Language • Team work

8. Except Australia and your country of origin, have you ever lived in any other countries? Have you ever formally studied Chinese? Where were you born? Branching or contingency questions Participants will be filtered towards branching or contingency questions applicable only to them (Creswell, 2008).

9. Handwriting tells you a lot about a person’s character. It is really important to get as much academic education as possible for one’s career. Compared to other families, mine is very wealthy. I can describe my past experiences in detail in Chinese. I feel completely comfortable at Chinese cultural events. I always go to venues, such as libraries, galleries, museums, theatres, and concerts, if they feature Chinese culture. I tend to mix exclusively with Chinese social groups. I can read Chinese newspapers and magazines easily. Children must be taught to paint, dance, or play musical instruments. Participants are “pushed” and split to select different responses.

10. Family members should live close to one another. I always prefer to live close to my family members. People always prefer to say yes to save face. To save face I always prefer to say yes. I am very popular among my Chinese peers. I am considered very popular among my Chinese peers. The answers to a question depend on, to a certain extent, whose opinions are under scrutiny.

11. I always keep up to date with current Chinese affairs by watching TV, listening to the radio, reading newspaper, and surfing online. I always keep up to date with current Chinese affairs by watching TV, listening to the radio, reading newspaper, or surfing online. I am well regarded because of my Chinese language competency. My Chinese competency is well regarded. Double barreled questions actually measure two things within one item.

12. Negatives in a statement may lead to errors in responses. I won’t have good time if I don’t celebrate Chinese festivals. I will have good time if I celebrate Chinese festivals. I feel very sad if I miss Chinese festivals.

13. Pilot study: Reliability and validity of the instrument • Sample size When significance level (α) and power value were set at .05 and .80 with a medium effect size of .30 as appropriate for a wide range of behavioral research (Hair, Black, Babin, Anderson, & Tatham, 2006), G-Power gave a sample size of approximately 60 to satisfy the Cronbach α test. • Sampling Convenience sampling

14. Main study • Sample size Although there is no single criterion that dictates the necessary sample size for Structural Equation Model, the sample size can be five times the number of indicators but no less than 200 in order to provide reliable results (Bentler & Chou, 1987). 46×5=230 • Sampling Snowball sampling (why not random sampling, systematic sampling, stratified sampling, cluster sampling?) • Data analysis Structural Equation Modelling

15. Key features of structural equation modelling • SEM is theory driven. • SEM is capable to assess how questionnaire items define their respective theoretical concept. • SEM improves statistical estimation by accounting for measurement error. • SEM is able to examine complex interdependence amongst theoretical concepts.

16. Chineseness Economic capital Cultural capital Social capital Symbolic capital I1 I1 I1 I1 I1 I1 I2 I2 I2 I2 I2 I2 I3 I3 I3 I3 I3 … … … … … … CHL proficiency e e e e e e e e e e e e e e e e e e

17. Cultural capital Chineseness CHL proficiency I1 I1 I1 I2 I2 I2 I3 I3 I3 … … … e e e e e e e e e

18. Thank You!