1 / 50

Implementing Survey Instruments in LOINC

Implementing Survey Instruments in LOINC. Thomas White, MD, MS. Outline. History and Vision Why are Survey Instruments Challenging? Proposed Migration to an Implementation Model (“the question is the concept”) Validation Nursing LOINC => Dialogix Dialogix Instruments => LOINC

tino
Download Presentation

Implementing Survey Instruments in LOINC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Implementing Survey Instruments in LOINC Thomas White, MD, MS Thomas White, MD, MS – NYS Office of Mental Health

  2. Outline • History and Vision • Why are Survey Instruments Challenging? • Proposed Migration to an Implementation Model (“the question is the concept”) • Validation • Nursing LOINC => Dialogix • Dialogix Instruments => LOINC • Lessons Learned • Next Steps Thomas White, MD, MS – NYS Office of Mental Health

  3. History • Motivation • Web-based instruments so can track symptoms (diagnoses), clinical outcomes over time • Aggregate across instruments / variants (e.g. severity of depression) • Challenges • 100s of instruments, >10K variables; no standards • Variants may not be comparable Thomas White, MD, MS – NYS Office of Mental Health

  4. Vision • National infrastructure for assessment instruments • Support all common instruments • Use a standardized coding system • Support multiple languages • Psychometric re-validation • Foster comparison of different ways of measuring construct Thomas White, MD, MS – NYS Office of Mental Health

  5. Outline • History and Vision • Why are Survey Instruments Challenging? • Proposed Migration to an Implementation Model (“the question is the concept”) • Validation • Nursing LOINC => Dialogix • Dialogix Instruments => LOINC • Lessons Learned • Next Steps Thomas White, MD, MS – NYS Office of Mental Health

  6. Variants May Not Be Comparable Original CNYPC Thomas White, MD, MS – NYS Office of Mental Health

  7. How Should Variants Be Coded? • Current approach • One code for all variants • Risks type I and II errors if scales differ • Safer approach • Unique code per minor variant • Reduces data sharing • Best approach • Fewest codes needed to disambiguate versions with truly different meanings or reference ranges • Catch-22 – research to compare versions requires separate codes Thomas White, MD, MS – NYS Office of Mental Health

  8. How Are Survey Variables Defined? • Conceptual definition • Construct being assessed • Operational definition • Wording of the question • Variable definition • Available answer options • Validation criteria • Associated internal coding values Thomas White, MD, MS – NYS Office of Mental Health

  9. What Can Modify a Variable’s Meaning or Reference Range? • How questions are posed • Wording and language • Context (order in which asked) • Presentation attributes • The method used to ask them • Likert scale, standard gamble, time trade-off, … • Who does the asking • Interviewer, self, computer, … • Who does the answering • Capability and willingness to answer truthfully Thomas White, MD, MS – NYS Office of Mental Health

  10. Wording • Section • Headers, instructions, labels • Question • Labels, instructions • Answer option • Labels, anchors, codes Thomas White, MD, MS – NYS Office of Mental Health

  11. Context / Ordering • Length • Waning attention or motivation • Proximity / Grouping • Neighboring questions affect expectations Thomas White, MD, MS – NYS Office of Mental Health

  12. Presentation • Look and feel of instructions / questions • Mechanism for asking / responding • Likert scales • Lengthy lists • Standard gamble, … Thomas White, MD, MS – NYS Office of Mental Health

  13. DISC: DiagnosticInterview Schedulefor Children More Complex Assessment Instruments ClinicalGuideline Scored Questionnaire StructuredDiagnosticInterview Thomas White, MD, MS – NYS Office of Mental Health

  14. Sources of Instrument Complexity • Branching (order / context) • Tailoring (wording) Thomas White, MD, MS – NYS Office of Mental Health

  15. A1 goto Q2 Q1 A2 goto Q3 A3 (implicit goto Q2) Q2 Q3 Branching: Traditional Solutions Rely on Post-conditioned Gotos Response-Level Branching (e.g. Perseus SurveySolutions) Hidden-Branching Nodes(e.g. NovaResearch’s QDS, Sawtooth Ci3) Q1 Hidden branching node if (exp) goto Q3 Q2 Thomas White, MD, MS – NYS Office of Mental Health

  16. Relevance–Based Branching (e.g. Dialogix, ?XForms?) no Q1: Q1.relevant? yes Ask Q1 Store N/A Q2: Q2.relevant? Ask Q2 Example: name: 1 What is the subject’s name? gender: 1 What is [name]’s gender? age: 1 How old is [name]? menarche: age > 9 && gender == ‘female’ Has [name] had her first period? LMP: menarche == ‘true’ When was [name]’s last period? sexual: 1 Is [name] sexually active? Relevance-based Branching Logic Thomas White, MD, MS – NYS Office of Mental Health

  17. A. SI DICE SI, ¿Hubo varias semanas seguidas en las que se sintió nervioso(a) o asustado(a) por estar separado(a) de [useted/su [FIGURA DE APEGO]] o fuera de su casa? Tailoring:Dynamically Composed Questions Some [children/young people] get very upset or nervous when they are not with their parents or with the grown-ups who usually look after them. I am going to ask you some questions about that. 1. In the last year – that is, since [[NAME EVENT]/[NAME CURRENT MONTH] of last year] – was there a time when [CHILD’S NAME] often wanted to stay at home and not go (to [school/work] or other) places without [you/[ATTACHMENT FIGURE]]? • A. IF YES, Where there several weeks in a row when [he/she] seemed nervous or afraid about being away from [you/[ATTACHMENT FIGURE]] or away from home? Thomas White, MD, MS – NYS Office of Mental Health

  18. Static vs. Dynamic Features • Instrument Definition (Static) • Wording & Presentation – untailored text • Order – branching logic • Instrument Use (Dynamic) • Tailored content • Actual order of questions seen • Time spent reading and answering • Who does the asking (face-to-face, self, …) • Modality used to ask it (Likert, standard gamble, …) Thomas White, MD, MS – NYS Office of Mental Health

  19. Implications for Coding Assessment Instruments • Meaning and reference range are: • Determined by operational and variable definitions • Altered by wording, order, presentation, assessment method • Further altered by dynamic tailoring, language, and actual order asked • Without separate codes, risk of type I and II errors • Need explicit modeling to compare variants Thomas White, MD, MS – NYS Office of Mental Health

  20. Outline • History and Vision • Why are Survey Instruments Challenging? • Proposed Migration to an Implementation Model (“the question is the concept”) • Validation • Nursing LOINC => Dialogix • Dialogix Instruments => LOINC • Lessons Learned • Next Steps Thomas White, MD, MS – NYS Office of Mental Health

  21. Current LOINC Schema for Nursing Instruments (1) Thomas White, MD, MS – NYS Office of Mental Health

  22. Current LOINC Schema for Nursing Instruments (2) Thomas White, MD, MS – NYS Office of Mental Health

  23. Current LOINC Schema – Limitations and Recommendations • Assumes one version of each instrument • Alter Method to create hierarchy of variants • Inadequate modeling of operational and variable definitions • Create formal XML model for these • No modeling of instrument branching logic • Extend LOINC Panel to codify instruments • No support for dynamic features • Can annotate answers/values with tailoring, branching, and timing information Thomas White, MD, MS – NYS Office of Mental Health

  24. Dialogix As Possible Solution • Syntax and schema for defining instruments • Each node represents a single measurable entity or communicates an important message • Engine for deploying them • Data Schema for storing results • Records order, logs backtracking and changed answers, stores fully tailored text and time spent / method used to answer questions. Thomas White, MD, MS – NYS Office of Mental Health

  25. Sample Dialogix Instrument Thomas White, MD, MS – NYS Office of Mental Health

  26. Dialogix Instrument Syntax Thomas White, MD, MS – NYS Office of Mental Health

  27. Dialogix Data Logs • Data (per answer given) • localName, answer, timestamp, sequence, tailored content, language used • Events (for each screen-full of questions) • Server, network, and browser processing time • Per-question keystroke and mouse events and timing Thomas White, MD, MS – NYS Office of Mental Health

  28. Four Recommendations for LOINC • Alter Method naming schema to disambiguate variants • Add Assessment_Instruments table • Migrate existing nursing instruments • Store instrument branching logic in LOINC Panel • Post-coordinate dynamic features with value portion of name-value pairs Thomas White, MD, MS – NYS Office of Mental Health

  29. (1) Alter Method Schema • [Observed|Reported].[Inst name].[Variant identifier] • Variant Identifier refers to entry in Assessment_Instruments table • Hierarchies • Root level has unspecified (or unknown method) • Levels below root specify details – initially one level deep (no known relationship between versions) • Re-arrange hierarchy as psychometric studies show similarity / differences among variants Thomas White, MD, MS – NYS Office of Mental Health

  30. (2) Assessment_Instruments Table • One row per measurable entity • Foreign Keys pointing to LOINC entry to text and LOINC Panel which codifies instrument • New Dialogix fields to fully codify instrument • Re-purpose Existing LOINC Columns • SurveyQuestionText => ActionPhase • AnswerList => AnswerOptions Thomas White, MD, MS – NYS Office of Mental Health

  31. Sample Concept Hierarchy Thomas White, MD, MS – NYS Office of Mental Health

  32. Sample Question – BPRS Thomas White, MD, MS – NYS Office of Mental Health

  33. Specifying Validation Criteria Thomas White, MD, MS – NYS Office of Mental Health

  34. Simple Calculations – Anxiety and Depression Subscale of BPRS Thomas White, MD, MS – NYS Office of Mental Health

  35. Complex Calculations: List of DSM-IV Diagnoses Thomas White, MD, MS – NYS Office of Mental Health

  36. Multilingual Support – XML Model Thomas White, MD, MS – NYS Office of Mental Health

  37. Multilingual – Translator’s View Thomas White, MD, MS – NYS Office of Mental Health

  38. (3) Store Instrument Sequence Information in LOINC Panel • Give each instrument variant its own Panel • Could use existing Dialogix XML schema • Issues • Consider using GLIF and Arden for metadata • W3C XForms, VoiceXML, and P3P schemas • Possible many-to-many relationship between questions and instruments • What about dummy variables? • What about module-level presentation features? Thomas White, MD, MS – NYS Office of Mental Health

  39. Dialogix Schema – Prototype for LOINC Panel Thomas White, MD, MS – NYS Office of Mental Health

  40. (4) Post-Coordinate Dynamic Features • Use ANO field within OBX of HL7 • Might use Dialogix data model • Should consider using a standard schema, like those in click-stream analysis. • Consider normalizing to XML-Event model Thomas White, MD, MS – NYS Office of Mental Health

  41. Outline • History and Vision • Why are Survey Instruments Challenging? • Proposed Migration to an Implementation Model (“the question is the concept”) • Validation • Nursing LOINC => Dialogix • Dialogix Instruments => LOINC • Lessons Learned • Next Steps Thomas White, MD, MS – NYS Office of Mental Health

  42. Columns in Assessment Instruments Table • One row per measurable entity • Instrument • Sequence • Concept • LocalName • SourceName • Relevance • ActionType • ActionPhrase • AnswerOptions Thomas White, MD, MS – NYS Office of Mental Health

  43. Validation: Existing Instruments LOINC => Dialogix • Concept = Component; Relevance = 1; ActionType = q • SourceName = Survey_Quest_Src • LocalName ~= Source Name or “q” + Sequence • ActionPhrase = Best of: • Survey_Quest_Text (e.g. LIV-HIV) • Comments (HHCC) • Component (Omaha, QAM, HIV-SSC) • Component: System (Omaha, reflexes in HHCC) • AnswerOptions = Best of AnswerList and Exmpl_Answers, using Dialogix’s syntax Thomas White, MD, MS – NYS Office of Mental Health

  44. Review Nursing Instruments in LOINC + Dialogix Thomas White, MD, MS – NYS Office of Mental Health

  45. Validation: New InstrumentsDialogix => LOINC (1) • Diagnostic Instruments, with Scoring • AUDIT, BPRS, BASIS-32, SF-36 • Structured Interviews • DISC, SCID-II • Decision Trees • GAF-Tree • Clinical Guidelines • NYC DOH TB Testing & Treatment Guideline • Healthcare Education • AutoMEQ Thomas White, MD, MS – NYS Office of Mental Health

  46. Validation: New InstrumentsDialogix => LOINC (2) • Most Axes Can Be Derived From Dialogix Schema • Property = FIND for questions; IMP for eqns • System = ^Patient (some exceptions) • Scale_Type = AnswerOptions and DataType • Method_Typ = [Observed|Reported].Instrument • Class = Survey.[Domain].Instrument • Some Axes Require Manual Review • Concept • Time_Aspect • Sometimes System, Method_Typ Thomas White, MD, MS – NYS Office of Mental Health

  47. Review Dialogix Instruments Newly Added to LOINC Thomas White, MD, MS – NYS Office of Mental Health

  48. Some Issues From Dialogix => LOINC • Modeling global and modular instruction? • Should any them be named? • Modeling equations – which should be named? • Add Date, Month, and Day to Scale? • Time_Aspect – add Lifetime when asking whether a diagnosis / symptom ever occurred? • Mapping local values to standard ones for dates, demographics, diseases, etc. • Supporting multiple languages Thomas White, MD, MS – NYS Office of Mental Health

  49. Conclusions • Instrument variants may need different codes. • Proposed LOINC extensions can • Model complex instruments • Disambiguate versions • Facilitate meta- and psychometric analysis • Operationalize instruments • Facilitate translation and ongoing QA • Additional modeling work needed for LOINC Panel and HL7 ANO attachments. Thomas White, MD, MS – NYS Office of Mental Health

  50. Thank You Thomas White, MD, MS – NYS Office of Mental Health

More Related