Assistive and Learning Applications for Intellectual and Developmental Disability

1. Development of the assistive and learning applications for people with Intellectual and Developmental Disability Marina Kultsova, Andrey UsovVolgograd State Technical University Software Engineering Department Volgograd, Russia

2. 2 Agenda • Brief overview of the research and educational activities of Software Engineering Department of Volgograd State Technical University. • Human-Computer Interaction for Special Needs: challenges and modern approaches. • Development of Adaptive Interfaces for People with Special Needs. • Presentation of our Projects: • Assistive service and mobile application for support of travel and communication for people with IDD; • Our partners - Volgograd Regional Rehabilitation Center for Disabled Children «Nadezhda»;

3. 3 Volgograd State Technical University • Volgograd State Technical University (VSTU) is the first base regional technical university in the Volga Region, preparing highly-qualified specialists in compliance with the multistage system of education - BS, MS and PhD. • VSTU consists of 3 affiliates, 3 representative offices, postgraduate and doctoral training programs, more than 20 000 students are trained in all forms of education. • The main activity of VSTU is to prepare the highly qualified engineering staff: chemists, metallurgists, architects, engineers, ecologists, economists on 35 specialties, 38 undergraduate areas and 21 areas of Magistracy.

4. 4 Software Engineering Department • Department was founded in year 2006. • Staff: 2 Professors, 4 Associate Professors, 3 Assistant Professors, 3 Assistants. • Students: more than 150 Bachelor students, 20 Master students, 7 PhD Students. • Educational Programs: • Bachelor Program “Software Engineering”, 4 years; • Master Program “Software Engineering”, 2 years.

5. 5 Software Engineering Department Research topics: • Software engineering methods and models, software design, testing and maintenance, software engineering management. • Artificial Intelligence and its application in intelligent support of decision making, IDSS. • Semantic Web Technologies and application in e-learning and open science, LMS, LMCS. • Human-computer interaction, development of dynamic adaptive user interfaces. • E-inclusion, assistive technologies and applications, human-computer interaction for special needs.

6. 6 Human-Computer Interaction for Special Needs • Adaptable and Adaptive Systems • Personalization • Acquiring Information about Users: • interaction behavior methods • probabilistic reasoning methods • plan recognition methods • machine-learning methods • clique-based (collaborative) filtering methods • clustering methods

7. Types of Information about the User ♦ Data about the user, such as demographic data, and information or assumptions about the user’s knowledge, skills, capabilities, interests, preferences, goals, and plans ♦ Usage data, such as selections (e.g., of webpages or help texts with certain content), temporal viewing behavior, user ratings, purchases and related actions, and usage regularities ♦ Environmental data, such as data about the user’s software and hardware environments, and information about the user’s current location and personalization-relevant data of this location

8. Adaptive User Interfaces User interface adaptations can take place in three areas of a system: (1) the selection of content to be displayed or recommended to the user, (2) the presentation of information including colors, font sizes, layout, etc., (3) navigation which defines the possible paths users can take through an application in order to access a certain information or functionality. Adaptive user interfaces which aim at improving the accessibility, however, will have to take into consideration all three design areas.

9. Requirements on Adaptive User Interfaces to Improve Accessibility Major challenges - creating design solutions that fit the individual users’ needs and finding adaptation mechanisms that lead to understandable and trustworthy system behaviour. The major requirements (Design for All): • Adapt content, navigation and presentation • Modular and extensible • Learn and adapt during run-time • Consistent and meaningful user interfaces

10. User Modeling in Adaptive Interfaces Semantic Modeling Use of Web Ontology Language (OWL) to represent semantic models of the user, the user-interface, the work domain, and the information content relevant to work in the domain. Resoning over ontologies as a major mechnism of adaptaion

11. A Design Patterns Approach • Four categories of design patterns: • generic patterns define global user interface settings to fit specific user needs and context conditions • interaction patterns provide suitable user interface elements and widgets for current interaction situations • common patterns define all the stable features of a user interface for a specific device which are not subject to adaptations, e.g. the resolution of the screen, a basic layout grid, etc. • transition patterns cover the dynamics of the adaptation processes, i.e. they define the mechanisms of switching from one generic or interaction pattern to another.

12. Travel and communication assistant for people with intelectual and developmental disabilities. • Project’s aim: increasing social mobilityand adaptation of people with mental disabilities. • Main functions: For people with mental disabilities: 1. Sending messages in pictograms or text. 2. Sending quick messages 3. Call For tutors of such people: 1. Messager’s part (messages and calls) 2. Making settings for own warded person 3. Travel assistant Now there is available connection between tutor and one person

13. Functional Requirements • Requirements of communication part • 1. Adaptive interface for people with disabilities • 2. Different input system for tutor: letters, pictograms • 3. Grouping people and getting messages from all person at group (available for tutors) • 4. Sending messages, quick messages between tutor and his warded person • 5. Calls between people • Requirements of travel assistant part • 1. Checking location of person with disabilities • 2. Creating a route with highlighted zones • 3. Notification about  track deviations

14. Legislation requirements • Development and testing problems • 1. Getting local position of warded person – problem with placing at Google Play/AppStore • 2. Testing by people with mental disabilities: • a) Special agreements from parents/tutor for working with such people and using our app • b) Travel assistant’s testing: hard to create realconditions for using it

15. 7 Device Technical Requirements • Operation system: • Android 4.0 or later • iOS 8.0 or later • Memory size - 512 mb • Free storage space - 500 mb • Display size - 4 in or more • Display resolution – 800x600 or higher • GPS or GLONAS

16. Tools and Technologies • Client side: • Cross-platform engine - Unity3d • Programming language - C# • Geographic location - Google Map • Server side: • Programming language - PHP • DBMS –MySQL • Communications protocol – XMPP • Other tools and data: • Version control system – Git • Translation service – Text2Picto • Used pictograms – Sclera, Pecs

17. Client-Server Architecture

18. Travel assistant prototype

19. Prototype

20. Adaptation for concrete user • Problems: • Can’t write and read all types of messages • Have problems with hands • Available: • Quick messages (prepared by tutor) • Calls to tutor

21. 8 Volgograd Regional Rehabilitation Center for Disabled Children «Nadezhda» • Center for Disabled Children "Nadezhda" is intended for complex rehabilitation of disabled children from 0 to 18 years old. • Child's stay at the Centre is up to 6 months. Daily the Center receives up to 200 children with disabilities. • The Center receives the children with disorders of the development of the central nervous system, musculoskeletal system, mental disorders, autism spectrum disorders, genetic diseases, including Down syndrome, somatic diseases.

22. 9 Rehabilitation Center «Nadezhda»

23. 10 Rehabilitation Center «Nadezhda»

24. Development of the assistive and learning applications for people with Intellectual and Developmental Disability Thank you for your attention!