Research Autism Lorna Wing Seminar Thursday , 2 4 th November 2011

1. Research Autism Lorna Wing Seminar Thursday, 24th November 2011

2. Does hand held technology improve independence? Helping young people with Asperger syndrome and high functioning autism navigate life’s highways. The ‘HANDS ‘project. An evaluation of multi site EU research collaborationJoseph Mintz London South Bank Unversity, LondonMiklos Gyori ELTE University & Autism Foundation, Budapest

3. Acknowledgements >HANDS project has been supported by a grant from the European Commission, within its 7th Framework Programme (Accessible and inclusive ICT; ICT-2007.7.2), contract nr. 224216. Authors are especially grateful for pupils’ and their families’ participation in the project.

4. Part I.HANDS:the project,the system,and its quantitative efficiency testing.Miklos GyoriELTE University, Budapest, Hungarygyorimiklos@elte.hu

5. the HANDS project

6. the HANDS project Helping Autism-diagnosed teenagers Navigate and Develop Socially June 2008 – October 2011 7th Frame Programme of the European Commission Accessible and inclusive ICT (ICT-2007.7.2) 10 partners from 6 countries

7. Stockholm, SWE Oslo, NOR Aalborg, DK London, UK Budapest, HU Cluj, RO

8. Stockholm, SWE ‘Svedenskolan’ School = Oslo, NOR ‘Helen Allison’ School + London South Bank University = Aalborg, DK London, UK ‘Egebakken’ School + Aalborg University = Cluj, RO Budapest, HU Autism Foundation School + ELTE University =

9. the HANDS project: aims a mobile-ICT-based support system for HF teenagers with ASD > specific aims >enhance social integration & participation >daily life skills, social behaviours & communication >via supporting well-established intervention approaches > further potential benefits >on efficiency of intervention >on efficiency of human resources >novel ways of data collecting/research

10. the HANDS system

11. the HANDS system two-component software system mobile component – for the client can be brought into focus situations of intervention individualised contents multimedia easy-to-handle web-based component – for the professional creating & managing contents ...even from away supervising mobile activity ...even from away sharing knowledge mobile  web remote synchronisation

12. >daily routines >institutional routines >social situations >transportation >free time >learning novel skills >etc.

14. an example >high functioning boy (12) with ASD > diabetes >swimming classes are vital but hard to manage for him

15. quantitative efficiency testingof the HANDS system key members of team:Ildikó Kanizsai-Nagy (Autism Foundation) Krisztina Stefanik (Autism Foundation & ELTE Univ.) Zsombor Várnagy (ELTE University)

16. developing & testing HANDS > 2-phase iterative development process >> 2 prototypes > 3 streams of scientific expertise & research > challenges and difficulties about development >quickly evolving technological context >efficient communication across disciplines > etc. > challenges and difficulties about testing & research >balance between development and research >balance between quantitative and qualitative methods >balance between involved research streams > cross-cultural, cross-institutional and individual differences

17. overall scheme oftesting HANDS Prototype 2 • > ‘quasiRandomised Controlled Trial’ • > test group: HANDS-assisted educational support > matched control group: ‘treatment as usual’ > pre-test / post-test & test / control comparisons • >limitations • > not totally random assignment • > no blind conditions > etc. • >active testing >from 15 January 2011 till 15 April 2011 • >at all the 4 test-schools

18. sample>4 countries, 6 schools, total N=49 (57) teenagers with ASD> male/female = 45/4>age: m= 14.38 years; SD=1.7; range 10,68 – 17,72>IQ: m= 85.37; SD=15.93; range 53 – 118 >diagnosis: clinical, confirmed by ADI-R and ADOS>educational context:autism-specific school or class with an evidence-based approachon a daily basis

20. HANDSPROTOTYPE 2 testing+ evaluation Psychology ELTE Univ. + Autism Found. testing + evaluation educational research LSBU testing+ evaluation human-computer interaction Aalborg University quantitativemethods qualitativemethods qualitativemethods

21. testing + evaluation psychology pre-assessment >>ADI, ADOS, IQ adaptive visual design >>eye-tracking quantitativemethods specific effects >>experimental task analysis skill-level effects >> psychometric testing usage >> ‘electronic footprints’ / log data

22. adaptive visual design >>eye-tracking > often atypical attentional processes in autism > eye-tracking/gaze-tracking: standard technique > method used here: ‘interactive-dynamic’ (developed by Gyori & Várnagy, 2010, 2011) >main finding: HANDS mobile is well-designed

23. specific effects >>experimental task analysis / ExTA > novel research tool developed by Krisztina Stefanik et al. (2010, 2011) > adapted from Task Analysis in TEACCH >combines quantitative measurement with individual focus > scales a target skill by the number & depth of necessary prompts >main finding: HANDS-assisted intervention is more effective than traditionalin a short run, in supporting specific adaptive behaviours

24. skill-level effects >> psychometric testing • >Social Responsiveness Scale, SRS,by Constantino & Gruber, 2005 • > 65-item questionnaire, 5 subscales, standardised • >HANDS Follow-up Questionnaire, by Gyori, Stefanik, Kanizsai-Nagy • > 55-item questionnaire • >both questionnaires • > parent/caregiver & teacher versions • > administered pre & post test • >statistical analysis > test/control, before/after comparisons • > key findings > ‘noisy’ results • > weak consistency between parents’ & teachers’ ratings • >lack of any sample-level effect

25. skill-level effects >> psychometric testing usage >> ‘electronic footprints’ / log data • > Data from SRS and HFQ questionnaires as above • + > Usage data recorded automatically by the HANDS system / ‘electronic footprints’ • >statistical analysis > correlations between usage frequency and effects • >key findings >huge differences in amount/frequency of usage • >positive correlations between usage and development in test period > cause  effect relationship?

26. summary of quantitative results skill-level effects >> psychometric testing usage >> ‘electronic footprints’ / log data • HANDS has adaptive, ‘autism-friendly’ mobile user surface. adaptive visual design >>eye-tracking • HANDS shows short-term efficiency of support in specific tasks. specific effects >>experimental task analysis • HANDS has no robust, general and specific long-term effect. skill-level effects >> psychometric testing • HANDS seems to have positive and long-term effects in a part of users.

27. conclusions from quantitative studies > on the HANDS support system >seems effective on both short and longer term… >in developing highly specific behaviours as well as wider skills > though not for all teenager-teacher user pairs >on conditions of success >> Part 2 > on quantitative research methodology > in spite of challenges & difficulties… > complex quantitative approach has proven productive and necessary >has been productively complemented by quantitative methods >> Part 2 > special thanks to > children and families… > schools and teachers…

28. Part II.the ‘educational research’ streamandqualitative studies on the HANDS systemJoseph MintzLondon South Bank Universitymintzjh@lsbu.ac.uk

29. Education: Research QuestionsApplicability of the HANDS tool to the learning environment > how HANDS fits in with existing practices of teaching and learning > what impact it has on such practices > effectsin developing social skills, self management skills and social integration > Identification of key factors influencing engagement and use

30. testing+ evaluation education Teacher interviews (mainly HA, but also EGE, SVE and AF) Parent and Child Interviews (HA only) qualitativemethods Class observations (mainly HA, but also EGE, SVE and AF) questionnaires (AF, EGE, SVE)

31. methods > interviews >semi-structured, pre-defined interview guide drawn from research questions >pre-test, in test and post-test period interviewing For Prototype 1: 7 teachers, 10 support staff, 8 parents, 9 children For Prototype 2: 9 teachers, 23 support staff, 6 parents, 10 children > questionnaires >39-item semi-structured questionnaire, matched to interview guide questions For Prototype 1: 14 questionnaires by teachers For Prototype 2: 16 questionnaires by teachers > observations >classroom based, matched to teacher interviews. >triangulation with interviews.

32. The Implementation Experience Technology can be hard to work with Software bugs – a mismatch in understanding about what “software testing” means Significant improvements in technology stability between Prototype 1 and Prototype 2

33. The Implementation Experience Technology can be hard to work with Connectivity issues Structural issues – can we get the permissions we need? Battery Issues, Charging Issues, Losing the Phone, Throwing the Phone Issues Teacher attitude and ability with ICT

34. The Implementation Experience But it can be transformational as well

35. The Implementation Experience Keeping up with the curve is hard Windows Dynamic Mobile, iPhone, Android, the Smartphone Revolution

36. The Implementation Experience Teenagers and Mobile Devices – what did we see for ASD? Independence, Adolescence, Group Identity, Mobile Loyalty – but more work is needed on this Jealousy, Bullying, Fear of Theft/Attack, Inappropriate Behaviour – text bullying – what did we see?

37. results / 1 > Developing children’s social and life skills >a mixture of successful and unsuccessful interventions >to aid a diverse range of skills >benefits and detriments of both HANDS software & general phone functions cited

38. It was difficult to arrange use of the device outside the classroom, partly on account of security issues, partly because of the software’s unreliability. (It is difficult to prepare in two ways for a possible problem. Eg., if behavioral aspects are only on the machine and it crashes, then the written form must absolutely be to hand.) These problems cause much less concern outside the classroom, since the structured setting is given there. [AF Teacher Questionnaire].

39. …..modification of the timetable was equally difficult, because the device must be connected to the computer for updates. [AF, 15156 Teacher Questionnaire]

40. Interviewer: Do you think he saw it as credible? Is it more than just playing with it, or is he responding to the interventions and doing what they tell him?Parent: Yeah, he definitely would, without a doubt, if he was out on his own, he would use it. if he got into a problem it would give him a great deal of confidence and it would focus him as opposed to him feeling the panic that I think he would feel if he did find himself on his own situation. [Parent Interview, HA]

41. Lucy: …we had parents evening, and he actually said that he wanted all his tasks put on the HANDS phone like that, so he didn’t have us nagging at him…. I went right okay, that means every teacher’s got to tell me what every task is in every lesson for me to put on there, but I have said to them that there are things that he does regularly during the week, so he goes to an offsite college on a Friday. He knows he has to have early lunch, come round to the department, get changed, be ready for when the minibus from the college pick him up and that. I said right we can put that on there. Don’t then tell him, nag him as he says… Just say to him check your HANDS phone, you know what’s expected, and just leave him to do it, and maybe that’s a way around it. Because Adam doesn’t like being told what to do [HA Teacher Interview]

42. results/2 Some key factors affecting engagement and use User Device Attachment Fogg (Fogg, 2003; Fogg and Eckles, 2007) proposes that the effectiveness of specific behavioural messages delivered via mobile devices is enhanced when there are ongoing repeated positive interactions with a range of cognitive and social functions on the device. Observed in 18 cases across PT1 and PT2 Important to note that not seen in all cases

43. results/2 Some key factors affecting engagement and use Source Credibility and a focus on the home Interviews and classroom observation showed that children can be positively disposed to receiving mobile persuasive interventions from their teacher, instantiated as a message from their teacher in video, image or text format. The present study provides evidence to support the exploitation of teacher credibility in the domain of life and social skills functioning. This very much part of the school curriculum for young people with ASD, yet in many instances the context for practising and developing social skills is as much in the home and in out of school contexts as within the classroom itself. Observed in 16 cases across PT1 and PT2. Important to note that not seen in all cases

44. results/2 Some key factors affecting engagement and use There was some evidence from Prototype 1 that instances of effective use of HANDS by children were predicated on the individual child recognising that an issue existed with a particular behaviour and having some level of motivation to engage with behaviour change. The Prototype 2 data provides further and stronger evidence to support this recommendation. All teachers in the UK school asserted in interviews that in their perception, student awareness of an issue or difficulty and associated motivation to achieve a specified behaviour change was an important factor mediating the level of engagement with HANDS and likelihood of successful response to a particular behavioural intervention instantiated on HANDS. In 4 out of 10 teacher-child cases at the UK school, teachers indicated that their student had difficulties with awareness of specific difficulties for which interventions were introduced on HANDS, and that this had in their perception been a contributory factor to lack of response to specific interventions, as shown in the following vignette involving John.

45. HANDS could potentially be used to encourage her student, John, a 15 year old male student with a diagnosis of autism, to reflect on his problematic behaviour with another student in the same class. John and the other student commonly tease and aggravate each other during lessons and break times. Teacher programmed a series of intervention prompts onto HANDS which asked John “has student X [the other student] had a good day?” which John could select to either answer yes or no, and then a further question “if you said no, was it because you wound him up?” which again John could answer either yes or no to. Intervention proved problematic, as John did not think that there was anything wrong with the way he socialised with student X or that his behaviour had any influence in the negative outcomes

46. After this intervention was implemented and John had been using it for a period, John requested that teacher change or stop the intervention because he did not like it, and instead the teacher rewrote the intervention, so that the reminder prompted John with “keep your joking with student X to break times”, a more directed intervention which relied more on leveraging teacher authority, and less on John’s internal awareness of the issue and motivation to change his behaviour. The teacher’s overall assessment of the intervention was that as John was not at all motivated to change the way he socialised and communicated with student X, the intervention failed to be effective.

47. results/2 Some key factors affecting engagement and use Preference for Mobile Device Based Interventions Our evaluation of Prototype 1 and Prototype 2 indicated that perceived teacher source credibility is a significant factor mediating engagement. We also had some weak indications from interview responses for Prototype 1 that some children may actually prefer to receive persuasive messages from HANDS than from their teacher. Accordingly we included specific questions on this area in our interview guide for Prototype 2 child interviews at the UK school. Six out of ten of the children commented on whether they preferred support / prompts from a person or from the HANDS device. Two students preferred prompts from a person, including parents, teachers and support staff. In contrast, the other four students at the UK school felt that they would rather receive persuasive messages from the HANDS phone than an adult.

48. results/2 Some key factors affecting engagement and use Preference for Mobile Device Based Interventions A potential rationale for this preference is that many children with ASD have cognitive impairments in processing speeds (Luna et al., 2007). For example, in one of the cases, the child was using HANDS to support him in the life skills task of making toast independently. In a classroom observation he is observed completing the task successfully using HANDS as a support and voices that he prefers to receive the instructions from HANDS as opposed to from his teacher. The teacher suggests that this may be because he “can do it in his own time”, that is that he has longer to process the individual intervention messages from the mobile device and can control the rate at which the messages are supplied. In the follow up interview the teacher comments:

49. I think they felt more relaxed because it was up to them to move to the next step, whereas you present someone with this long list of instructions written down. Someone with autism that you know, it works their minds when they see what they’ve got to do, because it does look a lot of set when it’s written down. Whereas on the HANDS phones they move to the next step when they felt confident enough to do so, and they worked through it didn’t they?[HA Teacher Interview]

50. conclusions & perspectives