CAI - Question Logic and Survey Content

1. CAI - Question Logic and Survey Content Ed Ross -OpenSurvey esr@opensurvey.org Jonathan Orgel - Kalonymous JOrgel@kalonymous.com Andrew Greenwald - Harris Interactive AGreenwald@harrisinteractive.com

2. AskML Object • Develop a semantic interface for CAI systems • XML Schema Based • For Exchange/transfer/process control

3. Current Survey System Architecture • Independent packages • Each package has authoring module(s) (script or GUI) and execution module(s) • Execution modules for each run on specific platforms (CAPI, Web, CATI, etc) • Proprietary forms of storing definition • No way of interchanging information other than at data level

4. A future survey architecture • Authoring: Blaise Quancept GUI2 Surveycraft • Transfer: XML • Execution: Blaise ASP’s Quancept WAP engine Documentation Export

5. Existing XML or text representations of CAI • DDI • TADEQ • US Census Bureau American Community Study • SPSS MR Core meta-data • Raosoft/Perseus Electronic Forms Language • SurveyUSA authoring/execution system • GIDE authoring/execution system • IDL representation of CASES instrument (not XML) • Triple-S XML • SIML survey engine from Knowledge Networks

6. Related Fields • VoiceXML • General language for speech /telephony enabling • IBM/Motorola/Lucent/AT&T • IMS Project • Interoperability of Computer Aided Training/Testing • ETS, DOD, etc

7. Some similarities and differencesQuestions – concepts similar, different wording What You Ask Based On Possible Answer Question Response Field Choice Variable question Category Question Choice Item Category Question Response Question Item Variable varinstance Category Variable Value

8. Different feature sets supported • Most support GoTo • Most support conditions on asking question(s) • 3 Support Limiting of answers by previous answers, etc • Most support loops. Others demand entries be repeated. • 2 Support shared response lists • 2 Support multilingual texts • 3 Provide information about data position in ASCII file • None support all of these

9. AskML Purposes • Agreed survey content representation • Common metadata format • Represent current practice - doesn’t force ‘best’ practice • Different execution engines in different technologies • Encourage interoperability • Allow authoring to be separated from delivery • Expedite sharing, re-use and library building • Enable general tools to document studies • Enable general tools to interface with metadata standards

10. AskML Design Goals • Usable With Existing Packages • Easy to Export • Preservation of Structure • Represent Information Needed for Authoring • Maximally Complete (Bigger is Better!) • Separate Content and Logic from Presentation • Multilingual • Multiversion

11. AskML Key Content • What are the Questions • What is the set of valid Responses for each • Which Questions are presented to a given respondent • Which Responses are presented to a given respondent • How are the Questions grouped for presentation • How do they actually appear on the screen

12. Proposed AskML Model SEPARATE LAYERS: • Questions • Logic • Formatting • Data Storage

13. Content Layer • Very few problems of conflicting content • Need agreement on nomenclature • Hope to achieve near 100% coverage • Will give ability to interface to archives, tab systems, etc

14. Content Layer • Questions (and blocks) defined as prototypes • Instantiated by use (“Ask”) • Response lists are separate entities • Blocks used for a variety of purposes • Logic • Loop • Single Screen Recommendation • etc

15. Content Layer - Questions • Questions do not have types - responses do • A question can have a collection of different response types within it • Complex questions, and grid questions thus are included

16. Content Layer - Questions • Example from American Community Study • “What is your usual bill for gas at home” • $_____ per month •  Gas is included in electricity bill •  Gas is included in rent •  Don’t know • (Even more interesting if you make it) • $______ per Month Quarter Year

17. Content Layer - Responses • Responses can have any of usual types • Categorical • Text • Numeric • Date • Currency • etc • Can also have ‘conversion’ numeric responses • Multi-currency • Units (eg weights expressed in kg, lb, oz)

18. Content Layer - Responses • Responses can have any number of aspects, some of which are ‘built-in’ • Whether response counts as ‘missing-data’ • Whether response has to be single selected • Permissible range for numerics, patterns for text • How responses are grouped if list is sorted/randomized • And some entirely user specifiable, with any criteria deemed useful, generally for selecting/categorizing • (eg - cars can have makers, styles, price bands, etc)

19. Content Layer - Responses • Responses can be specified directly or through referring to response lists • Response lists can themselves include response lists, allowing hierarchical constructions • Response lists can/should be shared between questions and between surveys

20. Content Layer - Texts • Texts can contain “fill”s for dynamic text substitution • Texts can contain tagged sections within them • Any text construct (eg “Question_Text”) can be a set of texts with different values of three aspects • Language (English, French, etc) • Purpose (Show respondent, show interviewer, use for statistical tabulations, show to respondent second consecutive time, use in table of contents, etc) • Medium (Use when final output is Web, when it is WAP device, when paper, etc)

21. Logic Layer - Routing • Major divide between packages based on GoTo logic and packages based on If/Then logic (or combinations) • AskML must internally be able to represent both modes • Conversion the responsibility of importing package or special tools • “Spaghetti” like instruments legal within AskML but not necessarily convertible

22. Logic Layer - Routing • Possible routing constructs to be saved/retrieved • If() Then….ElseIF…Else • If() GoTo • GoTo on specific responses • Precondition within question or block for it’s execution • (Future?) - Dynamic choice of questions - “just in time” asking

23. Logic Layer-Order/Selection • Need to provide limiting of presented responses, loops, etc based on previous answers or other information • Need to provide a range of cross question validation • Need to provide facility for ordering (randomize, rotate, alphabetize, etc) for • responses • questions • loop members

24. Logic Layer-Expression Syntax • Issue of whether to “tokenize” logical/arithmetic expressions, as in MathML • Or to accept/implement standard scripting syntax • if so, which? • JavaScript, Xpath, XFORMS DCL, Perl, etc… • AskML internal format is tokenized, based on MathML, with additions for survey specific features (which answers given, etc) • Will accept “escape” into scripting languages by specifying which scripting language and text of string

25. Logic Layer-Expression Syntax • <apply> • <scriptprocessor lang=”Javascript”/> • varA < varB + 5 • </apply> • could be used to represent • <apply> <!-- varA < varB + 5 --> • <lt/> <!--‘<’ or “less than operator” --> • varA <!-- 1st arg of <lt/> --> • <apply> <!-- Whole apply is 2nd arg of <lt/> --> • <plus/> <!-- ‘+’ or “addition” operator --> • varB <!-- 1st arg of <plus/> --> • 5 <!-- 2nd arg of <plus/> --> • </apply> • </apply>

26. Formatting Layer • Separate from content and logic • Any formatting within AskML is suggestive, not prescriptive • Ability to “suggest” simultaneous presentation • Allow external formatting systems - named templates within any template based system. • Likely increase in importance of XFORMS

27. Progress & Plans • Completion of Substantive Initial Spec in May 2001 • Issues: response limiting, logic syntax, etc • Draft XML DTD and Schema under development for Spring 2001 release • Preparation of documentation and examples • Development of parsers for Blaise, Quancept, QSL,Cases • Summer 2001 review of progress, next steps

28. Invitation • We welcome people who wish to participate in or comment upon any aspect of AskML • Contact any of the authors or • info@opensurvey.org