210 likes | 378 Views
Sampling for Surveys in the Dutch Statistical Bureau. Ida H. Stamhuis One of the chapters of the book The Statistical Mind in the Netherlands 1850-1940 (to be published in 2008) (co-authors Jelke van Bethlehem, Jacques van Maarseveen) For ‘the’ story I refer to that chapter
E N D
Sampling for Surveys in the Dutch Statistical Bureau Ida H. Stamhuis One of the chapters of the book The Statistical Mind in the Netherlands 1850-1940 (to be published in 2008) (co-authors Jelke van Bethlehem, Jacques van Maarseveen) For ‘the’ story I refer to that chapter Now I will suggest a more analytical approach Woudschoten, 28 and 29 September 2007
Continuation of my Woudschoten paper of 2005 Segmentation of various forms of statistics • Segmentation between ‘secret’ statistical practice of the public administration and statistical theory • Socio-economic-political explanations (didn’t go into that) • Segmentation within theoretical statistics • Two different forms of knowledge • Carried out by people with different ways of thinking • Carried out by people belonging to different intellectual cultures • No real communication between them
Vissering’s statistics representative of that of the ‘Statistical Movement’: qualitative-quantitative • Statistics as systematic description of all aspects of state / society • Statistics new discipline at the faculties of law
From the viewpoint of circulation of knowledge • Conclusion: segmentation; no circulation (for the time being) • It asks for an explanation. • In 2005 I concluded with the question:What kind of expertise is necessary? Result was discouraging • Another approach: Let us try out a theoretical social scientific notion and look at a related case in which the border has ultimately passed: ‘habitus’ of Bourdieu • Social experiences and circumstances result in ‘mental structure’ through which the world is experienced: habitus • Consequence: not easily changed by intellectual arguments, but rather by change of experiences and circumstances
Survey: observational study of social or economic factors of populations, emerged from the ‘Vissering’ statistical world • A sample is “a part for the whole”; a sample must be ‘representative’
19th century ‘monograph studies’ considered ‘representative’ • ‘typical entities’ • No exceptional cases • The whole is more than the sum of the parts
Kiaer: purposive samples representative • In purposive sampling: variation also taken into account • Around 1900 discusssion about the choice between complete enumeration and purposive sampling in the international statistical community: • Von Mayr: “One cannot replace by calculation the real observation of facts”
In England development of mathematically oriented (bio)statistics including random sampling; Pearson and Fisher • In random sampling expertise of the statistician is replaced by ‘blind’ chance • 1906 Bowley proposed random sampling in the English statistical community
1924 Commission in the International Statistical Community: report • Random and purposive selection both reason to exist • Describe meticulously the sampling procedure in each investigation • ‘run with the hare and hunt with the hounds’
Prominent Dutch Statisticians involved: • Verrijn Stuart: “I state that in principle no representative method, one or another, can have the significance of a complete enumeration of the phenomenon of study” • Otherwise random selection because of the ‘Law of Large Numbers’ • Methorst: purposive sample “helps to save a great deal of expense and labor”
Surveys mainly executed by official statisticians: educated in law, • directors of the CBS (Verrijn Stuart, Methorst, Idenburg), Van Zanten (director Amsterdam SB)
Dutch Statistical textbooks • 1910 Verrijn Stuart: • Monograph method • Purposive selection • 1927 Van Zanten: Law of Large Numbers • Purposive selection • Random selection • No preference between the two methods
Dutch Statistical textbooks • From the 1930’s people with more exact background invaded the Dutch statistical community: Accountants Bakker and Stridiron authors of textbooks, Tinbergen, Holwerda (actuary) • Bakker 1934: random sampling, probability theory in words. • 1939-2: In Foreword thanks to Methorst as well as Tinbergen and Holwerda • 1941-3: distinguishes between quality control and surveys • In distinct section Derksen introduces mathematical formula • Section about opinion polls
Dutch Statistical textbooks • Tinbergen 1936 Grondproblemen der Theoretische Statistiek (Basic Problems of Theoretical Statistics) • Random samples obvious • Accountant Stridiron 1941 Handboek der Bedrijfseconomische Statistiek (Handbook of Statistics of Business Economics): Mathematical parts written by Tinbergen • Random samples obvious • 1943-2 Derksen Co-author chapter ‘Samples”: Survey as well as Quality Control
Practice in CBS • Methorst tested in 1924 ‘representative method’ , partly purposive, partly random: it did not work • Then surveys for economic indicator’s like consumer price index; national expenditure surveys necessary: recruitment of households through labor unions and advertisements • Etc. Samples often purposive, sometimes mixed with random
Sampling in Market Research • Various organizations • Unilever established Lintas (1934) and IHO (1938) • NSS (1940) • NIPO (1945) • Sampling obvious: CBS director Idenburg felt in 1948 the need to explain why CBS chose for complete census • Sampling method not always clear • Problem of Non-response more attention than sampling method
Conclusions • Conceptual hurdles to go from completeness to samples: information is lost • Conceptual hurdles to go from purposive to random sampling: opinion of expert not relevant; mathematical principles not clear to the powerful statisticians in the first half of the twentieth century
Conclusions • Other hurdles: (“Socio-economic-political”) • Automation makes completeness cheap • Administration doesn’t like to fire officials • Complete material available • Organization structures are not easily changed
Why ultimately randomness accepted? • Not because statisticians start to understand the mathematical background • Rather because they become convinced of its relevance • Intermediaries (belong to both communities) play an important role • Other hurdles disappear