100 likes | 112 Views
This paper discusses selective editing strategies used by the U.S. Census Bureau for trade statistics programs. It explores different methods considered and the application of these methods on export transactions, including data adjustments, unit price analysis, and flagging records. The future work and challenges in evaluating the effectiveness of different score functions are also addressed.
E N D
Selective Editing Strategies for the U.S. Census Bureau Trade Statistics Programs María García, Alison Gajcowski, and Andrew Jennings U.S. Census Bureau UNECE Work Session on Statistical Data Editing, Bonn, Germany
Trade Statistics Programs: Background • Official source for the U. S. merchandise trade statistics • Monthly publications - Import/Export statistics - U.S. balance of trade • Collected by Customs and Border protection
Current Data Processing • Approximately 3.4 million import and 1.8 million export records per month • Edit master • Majority of edit failures are automatically imputed • Fewer than 0.5 percent of monthly records are “rejects”
Selective Editing • Prioritize manual review of edit failing records • Mandate: All rejected records are to be reviewed
Methods Considered • Flagging records • Effect of changes on totals • Adapted from Latouche, Berthelot (1992)
Methods Considered • Hidiroglou-Berthelot method (1986) - Applied to unit price ratios - Series of transformation on data - Use simple statistics • Hidiroglou-Berthelot & effect of changes on totals (Jäder and Norberg, 2005)
Application • Archived raw and “clean” data from 2004 export transactions • Data adjustments • Unit price • Totals, medians, quartiles, computed using the whole data set • Two types of weights • Data groups
Application • Flagging records – selected records that must be examined at the commodity level – 20% review level • Effect on totals, Hidiroglou-Berthelot – applied to data groups by commodities
Summary • Future work - Production testing, adjustment of weights, adding new weights - Review all rejected records? • Issue - How do we evaluate/compare the effectiveness of the different score functions?
Thank You! Maria M. Garcia U.S. Census Bureau Maria.M.Garcia@census.gov