1 / 10

The Norwegian CPI Data Validation and Editing

The Norwegian CPI Data Validation and Editing. 8-9 May 2008 Tom Langer, Statistics Norway. Survey systems – data capture. The CPI survey systems The regular surveys (40 pct) The special surveys (60 pct) Regular surveys Some 2000 outlets every month – 39 000 observations

priscilla-andrews
Download Presentation

The Norwegian CPI Data Validation and Editing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Norwegian CPIData Validation and Editing 8-9 May 2008 Tom Langer, Statistics Norway

  2. Survey systems – data capture • The CPI survey systems • The regular surveys (40 pct) • The special surveys (60 pct) • Regular surveys • Some 2000 outlets every month – 39 000 observations • No price collectors involved in data capture • Qualitative information added by respondent • Internet system ; 20 pct of respondents • Postal survey – questionnaire ; 80 pct

  3. Regular survey system – Some key figures

  4. Regular survey - validation • Step1 Initial cleaning • Likely decimal errors and key punch errors • Check against the questionnaire (electronic) • Step 2 Automatic flagging of observations • HB method combined with a normalised test • Decision criteria: An observation for further inspections should be flagged in both methods

  5. HB method set up • Basic test level: Regional product group (8 regions) • Fairly homogenous • Sufficient number of observations for robust estimation of median, quartiles • In some cases the number is too low • In case – system expands data set to cover all observations on national product level. • Test variables:T1 = pt / pt-1T2 = pt / pJuly

  6. Validation set up Transformation of the price relative distributions - in 2 steps: 1: Distributions symmetric around the median relative price 2: Allow for the influence of price levels U = 0,5 • Leads to the effect distributions for T1 and T2 Accept intervals according to HB method: • Lower Level = Em – C max (Em -Eq1;A Em) • Upper Level = Em + C max (Eq3- Em;A Em)

  7. The impact of A, U and C parameters

  8. Flagged extremes

  9. Editing Data received are edited in several steps • Initial cleaning of data • A second round based flagged extremes • Treatment of non response – automatic imputation Macro controls • Product level – region (8 regions) • COICOP level • A final impact control – top-down principle

  10. Special surveys – data capture • Cover 60 pct of the total CPI weight • Respondent burden • Respondents have well developed computer based systemsand are positive to share data • Surveys based on scanner data ; 30 pct of CPI weight • Food and beverages (300 000 obs) • Alcoholic beverages (14 000 obs) • New cars (1 750 obs) • Other surveys – administrative data

More Related