Two approaches to statistical inference

1. Two approaches to statistical inference “Bayesian approach” & “Frequentist approach”

2. Principle of Bayesian approach • Probability as a measure of personal uncertainty : degree of belief • All information presented with probability • Information combined and updated by using the rules of probability calculus • Most important output: posterior probability • state of information after seeing the observations

3. Properties of Bayesian approach • Can assess hypotheses of many kind • Prediction of new observations • Values of unobserved variables • Current, future and past events or propositions • Requires typically high computation power • -> started to gain popularity not before than 15 years ago • Controversial as it breaks the illusion of objectivity

4. Frequentist approach • Defines probability as “Limiting relative frequency of independent trials” • = Infinite number of trials : imagine that the data set would be collected for very large number of times while the state of nature remains unchanged • Tossing a bottle cap: the relative frequency of tosses in which the cap is upside down within infinite number of tosses. • There is no probability defined for the position of the cap in any specific toss • Frequency probability can be defined only to observables that are thought to vary because of random sampling: the true state of the world can not have frequency probability

5. Frequentist/classical/sampling approach • Three main areas • Hypothesis testing (p-values) • Point estimation • Interval estimation • Formal inference about hypothetical data: • How often one would get more extreme data if the assumed hypothesis was true and the sampling was infinitely repeated? • How would an estimator behave in future repetitions, if the estimate we have now was equal to the true value • Indirect inference about the state of the world: no measure of uncertainty

6. Which one to choose for a problem?