Composite scores based on Media use: special attention to Quality of Composite Scores and External Validation

1. Composite scores based on Media use:special attention to Quality of Composite Scores and External Validation Lluís Coromina Willem Saris Survey Research Center – ESADE Universitat Ramon Llull Barcelona Mini-Conference on Composite Scores (Barcelona) 14-15 February 2008

2. Overview • - Evaluation of ‘Total Time spent in the traditional media’. • - Evaluation of ‘Total Time spent on political issues in the traditional media’. • Evaluation of ‘Total Time spent for other purposes in the traditional media. • Design of the Media Use questions • Models Specification • Comparability and Quality of Composite Scores across countries and rounds • Comparison of effects (external validity) among models in Media Research

3. Newtons ideas about media use Knowing the total amount of media use: 1st) Whether people have access to a particular medium 2nd) Whether they actually make use of it 3rd) how much time people use different media (amount of media use) 4th) Purpose of its use Specific programs…. 5th) Sources of information Question design: Close-ended questions with large list of names of programs / specific programs Open-ended multiple choice design Close-ended multiple-choice questions with a list of the most important media purposes: politics, sport, entertainment….

4. Creation of Media measures Interests asked for the different media: Relative importance of the different purposes of use for each medium. The use of the media for different purposes at an individual level. • Traditional vs. New media Use of the news media in European Union Countries *Source: Eurobarometer

5. Questions from ESS main questionnaire A1. On an average weekday, how much time, in total, do you spend watching television? A2. And again on an average weekday, how much of your time watching television is spent watching news or programs about politics and current affairs? Questions also for listening Radio (A3, A4) and reading Newspaper (A5,A6)

6. Questions from ESS main questionnaire On an average weekday, how much time, in total, do you spend watching TV? • Method 1: - Method 2: WRITE IN HOURS: ANDMINUTES: - Method 3:

7. Questions from ESS main questionnaire • Transformation of the variables 1st ESS round

8. Questions from ESS main questionnaire • Transformation of the variables 2nd and 3rd ESS rounds

9. Creation of Indicators • Total Time spend in Media (TTM) = A1+A3+A5 • Total Time on Political Issues in the Media (TTPIM) = A2+A4+A6 • Total Time on Entertainment in the Media (TTEM) = (A1-A2)+(A3-A4)+(A5-A6) Where TTM = TTPIM + TTEM • A new relative measure: • Interest in Political issues in the Media (IPIMi) = (TTPIMi /TTMi)

10. Model Specification TTM= ‘TimeTV’ + ‘TimeRadio’ + ‘TimeNewspaper’ TTMi = Timei + i for i= TV, Radio, Newspaper E(δi) = 0for i= TV, Radio, Newspaper Cov(TTM, δ j) = 0 for all i Cov(Timei , δ j) = 0 for all i j Cov(δ i , δ I) = 0 when i j

11. Comparability of the Composite Score • There is no problem with comparability or equivalence because • TTM= S TTMi = S Timei + ifor i= TV, Radio, Newspaper • All variables in all countries are measured in the same units: minutes • The sum is also measured in minutes • There is a problem that the total time contains errors • So the quality has to be evaluated.

12. Quality of the TTM Composite Score • Quality of TTM = where in general • var(d) =S wi2 var(i) + 2S wiwj cov(ij) for i = j • If the errors are uncorrelated and the weights are 1 var(d) =S var(i)

13. Quality of the TTPIM and TTENT Composite Score Quality = • SB-MTMM Experiment was designed using TTM. • Proportional error in the different media with the same scale is the same. • Estimation of var(i) for TTPIM and TTENT

14. Table 4: Composite scores quality for ESS1, ESS2 and ESS3

15. External Validity

16. Measurement of dependent variables • Political knowledge: Number of times the respondents answered “don’t know” on questions concerning on political issues: Interpersonal discussion about politics: Frequency of political discussions: “How often would you say you discuss politics and current affairs?”. 7 point scale from “never” to “every day”. Political participation: Number of conventional actions done and number of protest actions done. Number of conventional actions done: “contacted a politician, government or local government official?”, “worn or displayed a campaign badge/sticker?” and “donated money to a political group”. Number of protest actions done is measured by whether or not the respondents “signed a petition”, “Taken part in a lawful public demonstration”, “boycotted certain products”, “deliberately bought certain products for political, ethical or environmental reasons” and “participated in illegal protest activities”.

17. “Media Total Time” models Fit criteria of the models: • To determine whether misspecifications are present (Saris & Satorra, forthcoming) • Procedure based on the size of MI , the power of the test for misspecifications and the EPC to determine whether a restriction is most likely incorrectly made.

18. TTPIM model Model for 20 countries in the ESS 1st Round

19. TTPIM model Effects TTM on the political variables are positive  more time spent on the media for political issues affects participation of the people, political knowledge and discussion. The effects from Newspapers are significant in all countries, except Ireland. In Austria, Finland, Portugal, France, Spain and Sweden the effects on political participation and discussion needed from reading political issues on the newspapers.

20. TTEM model Model for 20 countries in the ESS 1st Round

21. TTEM model Direct effects from TTEM on the political variables are all negative. The more time people spend on the media for Entertainment (non-political issues) the less they participate in politics affairs, discuss about political issues and know about politics. Additional Negative effects from television on discussion (Belgium, Finland, Germany, Ireland, the Netherlands and Switzerland). Additional Negative effects from television on political participation (Finland, Germany, The Netherlands, and Switzerland)

22. TTM model Model for 20 countries in the ESS 1st Round

23. TTM model • - There is no clear effect from TTM on the political variables  No pattern for the significant effects. • - Positive effects from newspapers on “political discussion” and “political participation”. • - Negative effects from television on “political discussion” and “political participation”. • Each medium has different direct significant effects. So the ‘total time spent in the media’ is not needed to explain several political variables. • Moreover, the TTM also requires additional positive effects from newspaper on political knowledge for 10 countries (not needed in the TTEM or TTPIM). • This can be explained because TTM is an aggregation of TTPIM and TTEM and therefore this variable presents a combination of the effects of the other variables

24. Conclusions - TTPIM and TTEM, despite some additional effects, can be interpretable and explaining the behaviour of political variables. - Using TTM, which is the sum of the other two, makes no sense, it cannot be used for determine the effects on political issues because it is a mixed of different interests or purposes. - Therefore, for media research we would suggest using the different types of media variables (instead a global measure of total time in the media) - It is very important in media research not only ask the total time spent on the media but to ask that for different purposes as is done in the ESS.

25. GRÀCIES PER LA SEVA ATENCIÓ!!! Lluís Coromina (lluis.coromina1@esade.edu) Survey Research Center – ESADE Universitat Ramon Llull Barcelona