1 / 75

Spatio-Temporal and Context Reasoning in Smart Homes

Spatio-Temporal and Context Reasoning in Smart Homes. Sook-Ling (Linda) Chua Stephen Marsland, Hans W. Guesgen. COSIT 2009. School of Engineering and Advanced Technology Massey University, New Zealand. T. h e S i t u a t i o n. . . . . The world is aging. - have we noticed?.

rane
Download Presentation

Spatio-Temporal and Context Reasoning in Smart Homes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spatio-Temporal and Context Reasoningin Smart Homes Sook-Ling (Linda) Chua Stephen Marsland, Hans W. Guesgen COSIT 2009 School of Engineering and Advanced Technology Massey University, New Zealand

  2. T h e S i t u a t i o n . . .     The world is aging - have we noticed? Source: United Nations (2007)

  3. T h e S i t u a t i o n . . .     • The populations of the world are aging Source: United Nations (2007)

  4. T h e S i t u a t i o n . . .     • The populations of the world are aging Source: United Nations (2007)

  5. T h e S i t u a t i o n . . .     • The populations of the world are aging > 25% < 10% Source: United Nations (2007)

  6. T h e S i t u a t i o n . . .     • People choose to stay in their ownhomes as long as possible and remain independent

  7. T h e S i t u a t i o n . . .     • People choose to stay in their ownhomes as long as possible and remain independent Aging leads to Physical disability Cognitive impairment • diminished sense and touch • slower ability to react • poor vision, hearing problems • memory problems

  8. T h e S i t u a t i o n . . .     • Supporting inhabitant’s daily activities • “Smart Homes” Figure extracted from:http://www.dreamhomesmagazine.com/

  9. T h e S i t u a t i o n . . .     • To react intelligently, the smart home needs to: (1) recognise inhabitant’s behaviour (2) perform reasoning • spatio-temporalinformation • contextual information

  10. B “Tokens” e h a v i o u r R e c o g n i t i o n     sensoroutput The Smart Home Figure extracted from:http://www.dreamhomesmagazine.com/

  11. B “Tokens” e h a v i o u r R e c o g n i t i o n     The direct representation of current sensor states being triggered E.g. of a sequence of tokens from the sensors

  12. B “Tokens” e h a v i o u r R e c o g n i t i o n     The direct representation of current sensor states being triggered E.g. of a sequence of tokens from the sensors

  13. B 19th Jan 2009 18:03:16 … 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 19th Jan 2009 19:37:21 19th Jan 2009 19:41:26 . . . . . . e h a v i o u r R e c o g n i t i o n     Office/Study Bedroom Laundry Dining/Living Room Kitchen Figure extracted from: The Aware Home, 2002

  14. B 19th Jan 2009 18:03:16 … 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 19th Jan 2009 19:37:21 19th Jan 2009 19:41:26 . . . . . . e h a v i o u r R e c o g n i t i o n     Office/Study Bedroom Laundry Dining/Living Room Kitchen Figure extracted from: The Aware Home, 2002

  15. B 19th Jan 2009 18:03:16 … 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 19th Jan 2009 19:37:21 19th Jan 2009 19:41:26 . . . . . . e h a v i o u r R e c o g n i t i o n     Office/Study Bedroom Q: How do we recognise behaviours? Laundry Dining/Living Room Kitchen Figure extracted from: The Aware Home, 2002

  16. B Office/Study Bedroom Laundry Dining/Living Room Kitchen e h a v i o u r R e c o g n i t i o n     • Challenges: (a) Exact activities are not directly observed, only the sensor observations 19th Jan 2009 18:03:16 . . . 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 19th Jan 2009 19:37:21 . . . . . . Figure extracted from: The Aware Home, 2002

  17. B Office/Study Bedroom Laundry Dining/Living Room Kitchen e h a v i o u r R e c o g n i t i o n     • Challenges: (a) Exact activities are not directly observed, only the sensor observations 19th Jan 2009 18:03:16 . . . 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 19th Jan 2009 19:37:21 . . . . . . Figure extracted from: The Aware Home, 2002

  18. B Office/Study Bedroom ? Laundry Dining/Living Room Kitchen e h a v i o u r R e c o g n i t i o n     • Challenges: (a) Exact activities are not directly observed, only the sensor observations 19th Jan 2009 18:03:16 . . . 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 19th Jan 2009 19:37:21 . . . . . . Figure extracted from: The Aware Home, 2002

  19. B e h a v i o u r R e c o g n i t i o n     • Challenges: (b) Same sensor activations will be involved in multiple behaviours

  20. B e h a v i o u r R e c o g n i t i o n     • Challenges: (b) Same sensor activations will be involved in multiple behaviours Figure extracted from: www.rebecca-waring.com, www.cyh.com, www.chow.com

  21. B e h a v i o u r R e c o g n i t i o n     • Challenges: (c) No. of observations can vary between activities Making breakfast Making dinner Microwave Oven Fridge Toaster Stove Cupboard Cupboard Drawer Tap

  22. B e h a v i o u r R e c o g n i t i o n     • Challenges: (d) Behaviours are rarely identical on each use • components can be present/absent • the order of individual components happen can change • length of time each piece takes can change E.g. Making a cup of tea With / without Milk / water first? How long?

  23. B e h a v i o u r R e c o g n i t i o n     • Challenges: (d) Behaviours are rarely identical on each use • components can be present/absent • the order of individual components happen can change • length of time each piece takes can change Stochastic Approach E.g. Making a cup of tea With / without Milk / water first? How long?

  24. B e h a v i o u r R e c o g n i t i o n     The Hidden Markov Model (HMM) • probabilistic graphical model • uses probability distributions to determine the states for a sequence of observations over time Source: Rabiner, L. (1989)

  25. B e h a v i o u r R e c o g n i t i o n     The Hidden Markov Model (HMM) • probabilistic graphical model • uses probability distributions to determine the states for a sequence of observations over time Observations We know this.. Source: Rabiner, L. (1989)

  26. B … … States But, not this e h a v i o u r R e c o g n i t i o n     The Hidden Markov Model (HMM) • probabilistic graphical model • uses probability distributions to determine the states for a sequence of observations over time Observations We know this.. Source: Rabiner, L. (1989)

  27. B e h a v i o u r R e c o g n i t i o n …     The Hidden Markov Model (HMM) … … States Observations • Markov property: • The probability of transition to a state (St+1) depends only on the current state (St) [represented by solid line] • The observation at Ot depends only on the state St at that time slice [represented by dashed line] Source: Rabiner, L. (1989)

  28. B e h a v i o u r R e c o g n i t i o n …     The Hidden Markov Model (HMM) … … States Observations • Markov property: • The probability of transition to a state (St+1) depends only on the current state (St) [represented by solid line] • The observation at Ot depends only on the state St at that time slice [represented by dashed line] Source: Rabiner, L. (1989)

  29. B e h a v i o u r R e c o g n i t i o n …     The Hidden Markov Model (HMM) … … States Observations • Markov property: • The probability of transition to a state (St+1) depends only on the current state (St) [represented by solid line] • The observation at Ot depends only on the state St at that time slice [represented by dashed line] Source: Rabiner, L. (1989)

  30. B Office/Study Bedroom Laundry Dining/Living Room Kitchen e h a v i o u r R e c o g n i t i o n     19th Jan 2009 18:03:16 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 . . . . . .

  31. B Office/Study Bedroom Laundry Dining/Living Room Kitchen Observations e h a v i o u r R e c o g n i t i o n     19th Jan 2009 18:03:16 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 . . . . . .

  32. B Office/Study Bedroom ? Laundry Coffee Machine Cupboard Fridge States Dining/Living Room Kitchen Observations e h a v i o u r R e c o g n i t i o n     19th Jan 2009 18:03:16 19th Jan 2009 18:07:56 19th Jan 2009 18:20:27 19th Jan 2009 18:33:44 19th Jan 2009 18:50:12 19th Jan 2009 19:01:08 . . . . . .

  33. B e h a v i o u r R e c o g n i t i o n     • To use HMM to recognise behaviours: (1)Segmentation break the token sequence into appropriate pieces that represent individual behaviours start end start end Observations . . .

  34. B e h a v i o u r R e c o g n i t i o n     • To use HMM to recognise behaviours: (2) Classification identify the behaviours using the HMM . . . Observations “Behaviour A” “Behaviour B”

  35. B e h a v i o u r R e c o g n i t i o n     Behaviour Recognition using HMM • Our approach: • Use a set of HMMs that each recognise different behaviours . . . “Making lunch” “Making coffee” “Showering” • These HMMs will compete to explain the current observations • Model selection is based on maximum likelihood Source: Chua, Marsland and Guesgen (2009)

  36. B e h a v i o u r R e c o g n i t i o n     Experiment: Competition between HMMs • Datasets • MIT PlaceLab • Designed a set of simply installed state-change sensors that were placed in two different apartments with real people living in them Source: Tapia (2004)

  37. B e h a v i o u r R e c o g n i t i o n     Experiment: Competition between HMMs • Datasets • The subjects kept a record of their activities that form a set of annotations for the data  “Ground-truth” segmentation of the dataset • We used the dataset from the first subject • 77 sensors • collected for 16 consecutive days

  38. B e h a v i o u r R e c o g n i t i o n     Experiment: Competition between HMMs • Datasets • Activities take place in one room (kitchen) • Location of the sensors is known a priori • Behaviours: • Prepare breakfast (toaster) • Prepare breakfast (cereal) • Prepare beverage • Prepare lunch • Do the laundry

  39. B e h a v i o u r R e c o g n i t i o n     Based on 727 observations (using 11 days testing and 5 days training set)

  40. B e h a v i o u r R e c o g n i t i o n     Based on 727 observations (using 11 days testing and 5 days training set)

  41. B e h a v i o u r R e c o g n i t i o n     Based on 727 observations (using 11 days testing and 5 days training set)

  42. B e h a v i o u r R e c o g n i t i o n     Experimental Results • Method works effectively • performs segmentation and detects changes of activities . . . observation Coffee Machine Drawer Drawer Microwave Fridge Cupboard Fridge

  43. B e h a v i o u r R e c o g n i t i o n     Experimental Results • Method works effectively • performs segmentation and detects changes of activities Preparing a beverage Preparing lunch . . . observation Coffee Machine Drawer Drawer Microwave Fridge Cupboard Fridge

  44. B e h a v i o u r R e c o g n i t i o n     • Discussion Lack of spatio-temporal information • Misclassification:  The end of one behaviour contains observations that should be the start of the next Preparing a beverage Preparing lunch … Drawer Coffee Machine Drawer Microwave Cupboard Fridge Fridge observation

  45. B e h a v i o u r R e c o g n i t i o n     • Discussion Lack of spatio-temporal information • Misclassification:  The end of one behaviour contains observations that should be the start of the next Preparing a beverage Preparing lunch … Drawer Coffee Machine Drawer Microwave Cupboard Fridge Fridge observation

  46. B e h a v i o u r R e c o g n i t i o n     • Discussion Lack of spatio-temporal information • Misclassification:  The end of one behaviour contains observations that should be the start of the next Preparing a beverage Preparing lunch … Drawer Coffee Machine Drawer Microwave Cupboard Fridge Fridge observation

  47. B Preparing a beverage Preparing lunch e h a v i o u r R e c o g n i t i o n     • Discussion Lack of spatio-temporal information • Misclassification:  The end of one behaviour contains observations that should be the start of the next Preparing a beverage Preparing lunch . . . Drawer Coffee Machine Drawer Microwave Cupboard Fridge Fridge observation

  48. B e h a v i o u r R e c o g n i t i o n     • Discussion Lack of spatio-temporal information Q: How to reduce misclassification? A: Augment current algorithm to include spatio-temporal information

  49. S p a t i o - t e m p o r a l     • Spatial information (Where?) • NOT directly interested in the exact coordinates • So, what are we interested in?  Room location  e.g. Figures extracted from: www.istockphoto.com, www.clubjam.jammag.com, www.nancilea.blogspot.com

  50. S p a t i o - t e m p o r a l     • Spatial information (Where?) • Current study used very basic spatial information (just the kitchen!) • In the future, . . .

More Related