1 / 21

Moving to a Centralized Database for Surveys in Blaise at NASS

Moving to a Centralized Database for Surveys in Blaise at NASS. Roger Schou National Agricultural Statistics Service IBUC XII Riga, Latvia. NASS – 46 Field Offices. NASS – 6 Data Collection Centers. Montana. Kentucky. Wyoming. Virginia. Oklahoma. Arkansas. Distributed Environment.

devi
Download Presentation

Moving to a Centralized Database for Surveys in Blaise at NASS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Moving to a Centralized Database for Surveys in Blaise at NASS Roger Schou National Agricultural Statistics Service IBUC XII Riga, Latvia

  2. NASS – 46 Field Offices

  3. NASS – 6 Data Collection Centers Montana Kentucky Wyoming Virginia Oklahoma Arkansas

  4. Distributed Environment • HQ-developed instrument typically distributed to 43 field offices • 43 Blaise datasets • Physical transfer of forms from Data Collection Centers to Client States • Survey management done in individual field offices

  5. Centralization Idea • Blaise 4.8.1 – Storing data in generic tables in a central database • Enable the Blaise data collection processes to be centralized • Splash of an idea to centralize one or two applications (CASIC and Survey Management System)

  6. Centralization Idea • Current Status of Development: • Too many stovepipe applications • Development vacuums • Lack of sharing code • Duplication of effort • Duplication of functionality • Management has the “I want it now” mentality

  7. Centralization Idea • Tidal wave - Centralization on an enterprise level • Blaise • Survey Management System • ELMO • Sample Design • EDR/EDC • PRISM • Analysis/Summary

  8. Database Decision • Choosing a database • Cost • Support • Commonality • NASS databases • Sybase • Redbrick • FoxPro • Oracle • MySQL ? Blaise

  9. Generic BOI Files • Blaise creates 8 tables • Blaise_Case • Blaise_Data • Blaise_Dictionary • Blaise_Form • Blaise_ID • Blaise_Key • Blaise_Open • Blaise_Remark

  10. Generic BOI Files • All surveys will be stored in a fixed, pre-defined structure • Translation tool to extract the data to our PRISM database (Sybase) will work for all Blaise data PRISM Blaise data Translation Tool

  11. Versioning • We plan on using Blaise versioning • Management made a requirement that original reported data must be preserved

  12. Folder Structures • At the onset we have to plan on allowing all field offices to run CATI • Currently planning to continue to use the Blaise call scheduler • The .BTR, .BOI, .BMI, .BDM, and .BXI files will be stored in unique folders for each field office (instrument and externals) • It would be nice to store the instrument files (and externals) in one common folder* \hqapps\casic\surveyname\ \data\casic\surveyname\fo\06\

  13. User Access Rights • Who can view and/or update the data? • Distributed – Physical Boundaries • Centralized – User Access Controls • Roles • HQ Statistician • Field Office Statistician • Supervisor Interviewer • Interviewer

  14. User Access Rights • A policy has been written and management approved • A team has been assembled to discuss the details and prepare for implementation

  15. Additional CASIC Tables • Information Tables • Survey Information Table • FIPS Allocation Table • User Information Table • Sample Master Table • Error Limits Table • Previously Reported Data Table

  16. Initializing the Database • Three Steps: • Survey Setup (HQ) • Folder creation • Initialize Preparation (FO) • Name & address files validated • Initialize (CRON / HQ) • Populate the database • Easily monitored by Survey Administrators • Problems easier to isolate

  17. Testing Plans • Scheduler Testing • Make sure that we did not introduce issues with the Blaise scheduler across field offices • Load Testing • Must support a maximum of 600 users simultaneously • Process Testing • What are the locking issues, if any, between processes

  18. Benefits • Real-time reporting of survey progress • Eliminating physical movement of forms • Bringing survey processes together • Seamless communication between systems • Eliminate “stove-pipe” applications • Sharing of code

  19. Challenges • Implementing user access rights • Dealing with Blaise datamodel changes • Coordinating • Interviewers across the country • Routing cases across call centers • Multiple modes of data collection • Other processes being centralized • Bridging the gap between applications during transition

  20. Conclusion • Exciting time of development • Positive attitude towards centralization • Anticipating the benefits to be gained • Management is realizing the large scope • Stepping back and realizing the development process (less of “I want it now”) • Communication is open and working well • Many processes evolving together

  21. Questions

More Related