1 / 43

Designing Survey Questionnaires with DDI Metadata

This proof of concept demonstrates how to design and run a survey questionnaire using DDI metadata, including organizing questions, capturing responses, and presenting the questionnaire.

thomaskkeller
Download Presentation

Designing Survey Questionnaires with DDI Metadata

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IMS 2.1.3Proof of Concept for Data Capture using Metadata Bryan Fitzpatrick Rapanea Consulting Limited June 2014

  2. Data Capture using Metadata • Aim is to demonstrate designing and running a survey questionnaire based entirely on metadata • Aim is to use DDI metadata • design the questions • organise the questions into a questionnaire • present the questionnaire • capture and save the responses • all based entirely on the metadata

  3. Using DDI Metadata for Questionnaires • DDI has metadata for Questions • a simple question goes in a Question Item • What is your age in years? • a complex question goes in a Multiple Question Item • Did you do paid work last week? • Full Time or Part Time? • How many hours? • A Multiple Question Item can contain Question Items or other Multiple Question Items

  4. Using DDI Metadata for Questionnaires • Questions can link to one or more Concepts • to indicate what the question is seeking to cover • Age, Sex, Country, Income, Occupation, ... • perhaps to qualify what is being covered • eg Non-farm income, Tertiary qualifications

  5. Using DDI Metadata for Questionnaires • Questions have: • Name • just a multi-lingual name, not used in questionnaires • Text • the question that is asked • can be conditional, multi-lingual, formatted • can even have mixed language • Question Intent • some elaboration about what is being sought • multi-lingual, formatted • POC just uses simple unformatted multi-lingual Text

  6. Using DDI Metadata for Questionnaires • Questions have Response Domains • what sort of answer is expected or valid • Numeric domain • can specify integer of decimal, valid formats and ranges, etc • Text domain • can specify format, length • Category Domain • valid list of multi-lingual values • not really very much use • Code Domain • valid list of multi-lingual values with codes • a classification

  7. Using DDI Metadata for Questionnaires • Questions have Response Domains • what sort of answer is expected or valid • Date-Time Domain • can specify formats • Geographic • eg coordinates, other units • Structured Mixed Response Domain • a combination of all of the above • all domain type can have labels and descriptions

  8. Using DDI Metadata for Questionnaires • Questions do not go directly into a questionnaire • DDI calls a questionnaire an Instrument • questions constitute a library available for use • a “Question Bank” • questions are selected and assembled into an Instrument • the assembling of questions is done with Control Constructs • an Instrument identifies a single Control Construct that builds the questionnaire

  9. Control Constructs • Control Constructs are the critical component in building a questionnaire • they select the questions • they control the flow of the questions • branching and looping • they insert non-question text • “Now I want to ask you about other people in the household” • they can compute values • they link to Interviewer Instructions • structured DDI Interviewer Instructions • unstructured external interviewer instructions material

  10. Control Constructs • Several types of Control Constructs • Question Construct • selects a Question Item or Multiple Question Item • Sequence • selects a sequence of other control constructs of any type • If-Then-Else • defines an If condition with optional ElseIf clauses (multiple) and optional Else clause • each condition selects a single Control Construct to include

  11. Control Constructs • Several types of Control Constructs • Loop, Repeat-Until, Repeat-While • eg to loop over people in a household • Statement Item • inserts non-question multi-lingual text (conditional, formatted) • Computation Item • a calculation in some language that is assigned to a Variable

  12. Instrument • Identifies a single Control Construct to assemble the questionnaire • probably a Sequence construct • Instruments can have an Type • a single value taken from some Controlled Vocabulary • a user-managed list of valid values • eg, Paper, Internet, CATI, ... • Instruments can have multiple Software specifications • basically just identifying “software” used with instrument • not a great deal of use

  13. Instrument • Instruments do not have any place for useful layout metadata • just the type of the layout • a fairly serious limitation • We need quite a lot of information to do the layout • how to represent lists • tick boxes, list boxes, combo boxes, radio buttons • how to show flow logic • which questions to show at once, which to separate • can the respondent backtrack • We need additional Layout Metadata • I have designed some

  14. Interviewer Instructions • A formal DDI metadata type • Organised, structured instructions • formatted multi-lingual text • may be conditional • May link to external, non-DDI material • eg, PDF, Word documents • Not used in this Proof of Concept

  15. Classifications • DDI holds Classifications as linked Code Schemes and Category Schemes • a Category Scheme is a list of Categories • flat list of multi-lingual names and descriptions • eg, Country names, Occupation names, etc • a Code Schemes selects Categories from Category Schemes, assigns a Code (not multi-lingual), and may specify a hierarchy • a Code Scheme may select Categories from multiple Category Schemes • multiple Code Schemes may select the same Categories

  16. Code Schemes and Category Schemes • Used for • Classifications • a Classification is a Code Scheme • Controlled Vocabularies • lists of standardised terms • defined by DDI, an organisation, a local area

  17. Code Schemes and Category Schemes • Used for • Response Domains for Questions • Code Domains and Category Domains • Category domains are not much use in a multi-lingual environment • Categories have different names in different languages • with no unique handle except a meaningless Id • Representations for Variables • Code Representation

  18. Variables • A Variable is a container that will hold a data value • has a Name and Description (both multi-lingual) • can be linked to a single Concept • to indicate what the data represents • can be linked to multiple Questions • to indicate where the data comes might come from • can have a Representation • Code, Date/Time, Numeric, Text • with constraints on values • can identify a Response Unit and an Analysis Unit • a population that it can apply to

  19. Logical Record • A Logical record consists of a sequence of Variables • groups data values for a purpose • data from a questionnaire goes into one or more Logical Records • Logical Records can be linked • eg, Households and Persons • Logical Records are independent of any storage or stored format

  20. Record Layouts and Physical Structures • Map a Logical record to a physical record and an actual stored file format • Can support a very wide range of structures and storage formats • CSV, Binary file, XML, database • multiple record types, linkages of many kinds • POC does not actually use this • Simple CSV file maps directly from Logical record

  21. Physical Instance • Holds information about actual data sets produced • links to Physical Structures, Record Layouts, and Logical records • provides a central management of data from a collection • POC uses Physical Instance to manage data • POC 2.3.3 builds on this POC to show how to use SDMX and DDI metadata together • produces tables from SDMX DSD using data collected with DDI • uses the Physical Instance information to find the datasets

  22. What does the POC do? • Collects survey data based entirely on metadata • builds or imports all the metadata • assembles a survey instrument (questionnaire) • presents the questionnaire in a Windows Form • collects data into Logical Records • saves the data in CSV files • POC 2.3.3 builds on this POC • duplicates Concepts and Classifications to SDMX • tightly-coupled set of metadata • uses SDMX DSD to produces tables from the collected data

  23. How does the POC do This? • Basically POC system is a metadata creator/editor • Build, import Concepts • build in UI, import from CSV, SDMX V2.0 • Import Classifications • import CSV, SDMX V2.0 • did not implement build in UI • Construct Questions in UI • multi-lingual text with links to Concepts • POC almost supports Multiple Question Items • by the end of the week

  24. How does the POC do This? • Build Variables • links to Concepts and Questions • Build Control Constructs • Question constructs • link to a question • Sequence constructs • define a sequence of other Control Constructs • If-Then-Else constructs • allows conditional questionnaire flow • POC does not support Loop, Repeat-Until, Repeat-While, Computation Item, and Statement Item constructs

  25. How does the POC do This? • Define Instruments • link to a single Control Construct • probably to a Sequence construct • has an instrument type • Windows Form, Web, Paper, CATI, .. • POC only supports Windows Form • Define Logical Records • collection of Variables • Map Logical record to Instrument • map Variables to Questions • uses Concept links and question links if present • allows user override in UI

  26. How does the POC do This? • Present the Instrument • Render the instrument in a Windows Form • using some layout metadata I made up • Execute the Control Constructs to select questions and manage question flow • present questions in list box, combo box, radio buttons, text box • depending on response Domain and some Layout Metadata • not DDI metadata, my design • capture the responses into a Logical record • based on the Logical Record – Instrument mapping • present questions in language of choice • limited choice

  27. How does the POC do This? • Retrieve Logical Record set • run interview multiple times to get a set of logical records • results displayed on screen in tabular form • mapping as defined in Response Domains and Logical record to Instrument map • Save the Logical records in data file • CSV file • no actual Record Layout and Physical Structure metadata • simplest Record Layout and Physical Structure metadata is almost empty anyway • it is just saving the Logical Records with Variables separated by commas

  28. How does the POC do This? • Save details of CSV data sets in Physical Instance metadata • so the data can be found for subsequent operations • like producing tables in POC 2.3.3

  29. Metadata is in a DDI Instance file DDI Instance Layout Metadata file Group Layout Metadata Concepts Code Schemes and Category Schemes Questions Control Constructs Instruments Variables Logical Records Study Unit Physical Instance

  30. DDI is fairly complex • But so is designing a Questionnaire! • Constructing questionnaire involves constructing a lot of metadata • but DDI is fairly logical • you need to think about what you want in the questionnaire and how you want it to flow • but you need to do this if you are designing a questionnaire manually • once you design questionnaire you get re-use advantages • easy to modify, add languages • easy to adapt to some other purposes • easy to reuse useful questions and constructs • easy to capture the data

  31. Let us look at my test questionnaire • Simple questions about internet access • based on Eurostat ICT survey • includes two If-Then-Else constructs to manage flow • we will look at the structure first • a good way to plan your own questionnaire • a good way to see how the DDI metadata works

  32. ICT question flow 1 Country 2 Sex 3 Age A1 Do you have access to a computer at home? A2 Do you have access to the internet at home? B1 When did you last use a computer? If (within last 3 months) B2 How often on average? C1 When did you last use the internet? If (within last 3 months) C2 How often on average?

  33. ICT question flow 1 Country ------- QC 1 2 Sex ------- QC 2 3 Age ------- QC 3 A1 Do you have access to a computer at home? ------- QC 4 A2 Do you have access to the internet at home? ------- QC 5 B1 When did you last use a computer? ------- QC 6 If (within last 3 months) B2 How often on average? ------- QC 7 C1 When did you last use the internet? ------- QC 8 If (within last 3 months) C2 How often on average? ------- QC 9 QC – Question Construct

  34. ICT question flow 1 Country ------- QC 1 2 Sex ------- QC 2 3 Age ------- QC 3 A1 Do you have access to a computer at home? ------- QC 4 A2 Do you have access to the internet at home? ------- QC 5 B1 When did you last use a computer? ------- QC 6 If (within last 3 months) B2 How often on average? ------- QC 7 If C 1 C1 When did you last use the internet? ------- QC 8 If (within last 3 months) C2 How often on average? ------- QC 9 If C 2 QC – Question Construct If C – If-ThenElse Construct

  35. ICT question flow Seq C 1 1 Country ------- QC 1 2 Sex ------- QC 2 3 Age ------- QC 3 A1 Do you have access to a computer at home? ------- QC 4 A2 Do you have access to the internet at home? ------- QC 5 B1 When did you last use a computer? ------- QC 6 If (within last 3 months) B2 How often on average? ------- QC 7 If C 1 C1 When did you last use the internet? ------- QC 8 If (within last 3 months) C2 How often on average? ------- QC 9 If C 2 QC – Question Construct If C – If-ThenElse Construct Seq C – Sequence Construct

  36. Let us have a look at the POC • Demo

  37. What does the POC show?

  38. What does the POC show? • It is realistic to use the DDI metadata to design and present a survey questionnaire • The POC depended entirely on the metadata • absolutely no knowledge about the survey built into the system

  39. What does the POC show? • It is realistic to use the DDI metadata to design and present a survey questionnaire • The POC depended entirely on the metadata • absolutely no knowledge about the survey built into the system • Designing a survey questionnaire in DDI is fairly complex • So is designing one by manually

  40. What does the POC show? • It is realistic to use the DDI metadata to design and present a survey questionnaire • The POC depended entirely on the metadata • absolutely no knowledge about the survey built into the system • Designing a survey questionnaire in DDI is fairly complex • So is designing one by manually • There are some clear advantages • modification and reuse is easy • multi-lingual presentation is easy

  41. What does the POC show? • Really need some form of Layout Metadata • DDI Instrument and Control Construct metadata gives no guidance on layout • POC case was very simple • but still needed some layout information • realistic survey questionnaire needs considerable layout information • needs more thought to design this metadata

  42. What does the POC show? • POC used a Windows Form questionnaire • no practical use for production survey • but other questionnaire formats should be easy • hope to have paper example (in Word) by end of week • script for a Web Form questionnaire is straight-forward • but Web Form system still needs to process Control Constructs • script for CATI is easy • script for Blaise should be easy • easy to have questionnaire available in multiple formats

  43. Thank you • Questions? • Bryan FitzpatrickRapanea Consulting LimitedBryanMFitzpatrick@Yahoo.CO.UKPh +44-7789-886536

More Related