Measurements, Meaningfulness and Scale Types

Measurements, Meaningfulness and Scale Types Simon Frenchsimon.french@warwick.ac.uk

A few thoughts to begin with • Kate (aged 6): “You cannot have a ‘5’, can you, Dad? You have to have 5 things.” • Would you average the numbers on a few car number plates? • REF provides a perfect way of measuring my research ability • I am overweight …

Numbers have mystique … • … especially to scientists! • “…when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.” Lord Kelvin • Ferguson Report of the BAAS 1932: • “…any law purporting to express a quantitative relation between sensation intensity and stimulus intensity is not merely false but is in fact meaningless unless and until a meaning can be given to the concept of addition as applied to sensation.” • Essentially if you cannot put a number to it which you can add, subtract, kick around with arithmetic, it isn’t science

Statistics and Numbers • Well , they are the same thing, aren’t they? • Without numbers, where would statistics be? • So lets unpack the meaning of numbers in ‘measurement’ and statistics

Representation by Numbers • Possible relations between objects: • Just recognisable as separate objects • To the right of • Taste nicer • Hotter then • Rounder than • Heavier than • ….. The ‘World’ Objects in the World Measurement:Attaching numbers to represent the objects in such a way that numerical relations between the numbers mimic the relations between the objects But once we have numbers we can do lots of arithmetical things with them. Is it meaningful to do so?

UGA of Transarcadia Award teaching funds according to mean matriculation grade Change grades to numbers: A → 5, B→4, C→3, D→2, E→1. Rosenblock mean grade: 3.3. Altenburg mean grade: 3.2 Instead associate grades with minimum mark for grade: A → 90, B→75, C→65, D→50, E→30 Rosenblock mean grade: 65. Altenburg mean grade: 66 The ranking of means of ranking measurements is meaningless It depends on the ‘spurious’ choice of numbers to represent the ranking

Reliability, Validity and Meaningfulness In statistical analysis need to consider • Validity • Are we measuring what we think we are • Reliability • If we measure it again and again, do we get the same measurement • Meaningfulness • When can we use the numbers arithmetically in ways that reflect the world out there?

Scale types

Issues for statistics • Many statistical methods require data measured on interval or ratio scales • in theory at least • pragmatically they may work on ordinal data … if you do not believe p values, significance levels, etc. • There are methods for ordinal data • e.g. many nonparametric methods.

Naudeet al (2010) MBA rankings • Naude, P, Henneberg, S and Jiang, Z (2010) ‘Varying roots to the top: identifying different strategies in the MBA marketplace’ Journal of the Operational Research Society 61, 1193-1206

Context • MBAs are judged according to • Various criteria • Top criteria: ‘Career progress’, ‘diversity’ and ‘idea generation’ • Accreditation • Data • FT rankings of top 100 MBAs, each assessed against a number of criteria, some ordinal, some interval. • So it would be dubious to use any analysis that assumes that all criteria are measured on interval or ratio scales

Initial analysis • Table 1, p1195 • Simple grouping to look at high level behaviour • Uses means so ‘assumes’ everything on interval or ratio scales • Check things make some sort of rough sense • Check with ANOVA roughly …. • Again assumes interval or ratio scales • Then with discriminant analysis • Plot in three dimensions and find planes that separate the MBAs into clumps • Again assume interval or ratio scales • But only getting a ‘feel’ for the data • Seeing what makes ‘sense;

Cluster Analysis (Table 2, p 1196) • Takes a measure of distance between two MBAs (e.g. Euclidean) • Find clusters of MBAs that are close together • But checked with other distance measures • Little/no effect on clusters

Multidimensional scaling (Figure 1, p1198) • Each MBA represented by 8 sub-criteria of career progress. • Try and plot in two dimensions so that 2-d distance between MBA reflects distance between them in full 8- dimensions • Again the plots do not vary much if different distance measures used

Factor Analysis (p1198) • Diversity is measured by 9 different sub-criteria • Can we reduce the dimensions • Again factor analysis is geometric (rotating axes to find the one that has the most variation) so assumes interval scales • But develops two components (Table 5)

Cluster Analysis (p1200) • This time on ‘idea generation’

Naudeet al’s Analysis • Exploratory • Multiple runs with different ‘spurious’ choices • Sensitivity analysis • Use p-values and significance levels as indications only • Looked for common themes and qualitative understanding • Their analyses ‘triangulate’

Theory of Measurement • Stevens: Scale types • Krantz, Luce, Suppes and Tversky: Theory of Measurement Vols 1, 2 and 3. • Very mathematical • F.S. Roberts (1979) Measurement Theory. Vol 7 in the Encyclopaedia of Mathematics Cambridge University Press.

Measurements, Meaningfulness and Scale Types