Intelligent Environments

1. Intelligent Environments Computer Science and Engineering University of Texas at Arlington Intelligent Environments

2. Prediction forIntelligent Environments • Motivation • Techniques • Issues Intelligent Environments

3. Motivation • An intelligent environment acquires and applies knowledge about you and your surroundings in order to improve your experience. • “acquires”  prediction • “applies”  decision making Intelligent Environments

4. What to Predict • Inhabitant behavior • Location • Task • Action • Environment behavior • Modeling devices • Interactions Intelligent Environments

5. Example • Where will Bob go next? • Locationt+1 = f(…) • Independent variables • Locationt, Locationt-1, … • Time, date, day of the week • Sensor data • Context • Bob’s task Intelligent Environments

6. Example (cont.) Intelligent Environments

7. Example • Learned pattern • If Day = Monday…Friday & Time > 0600 & Time < 0700 & Locationt = Bedroom Then Locationt+1 = Bathroom Intelligent Environments

8. Prediction Techniques • Regression • Neural network • Nearest neighbor • Bayesian classifier • Decision tree induction • Others Intelligent Environments

9. Linear Regression Intelligent Environments

10. Multiple Regression • n independent variables • Find bi • System of n equations and n unknowns Intelligent Environments

11. Regression • Pros • Fast, analytical solution • Confidence intervals • y = a ± b with C% confidence • Piecewise linear and nonlinear regression • Cons • Must choose model beforehand • Linear, quadratic, … • Numeric variables Intelligent Environments

12. Neural Networks Intelligent Environments

13. Neural Networks • 10-105 synapses per neuron • Synapses propagate electrochemical signals • Number, placement and strength of connections changes over time (learning?) • Massively parallel Intelligent Environments

14. Computer vs. Human Brain Intelligent Environments

15. Computer vs. Human Brain “The Age of Spiritual Machines,” Kurzweil. Intelligent Environments

16. Artificial Neuron Intelligent Environments

17. Artificial Neuron • Activation functions Intelligent Environments

18. Perceptron Intelligent Environments

19. Perceptron Learning Intelligent Environments

20. Perceptron • Learns only linearly-separable functions Intelligent Environments

21. Sigmoid Unit Intelligent Environments

22. Multilayer Network ofSigmoid Units Intelligent Environments

23. Error Back-Propagation • Errors at output layer propagated back to hidden layers • Error proportional to link weights and activation • Gradient descent in weight space Intelligent Environments

24. NN for Face Recognition 90% accurate learning head pose for 20 different people. Intelligent Environments

25. Neural Networks • Pros • General purpose learner • Fast prediction • Cons • Best for numeric inputs • Slow training • Local optima Intelligent Environments

26. Nearest Neighbor • Just store training data (xi,f(xi)) • Given query xq, estimate using nearest neighbor xk: f(xq) = f(xk) • k nearest neighbor • Given query xq, estimate using majority (mean) of k nearest neighbors Intelligent Environments

27. Nearest Neighbor Intelligent Environments

28. Nearest Neighbor • Pros • Fast training • Complex target functions • No loss of information • Cons • Slow at query time • Easily fooled by irrelevant attributes Intelligent Environments

29. Bayes Classifier • Recall Bob example • D = training data • h = sample rule Intelligent Environments

30. Naive Bayes Classifier y represents Bob’s location • Naive Bayes assumption • Naive Bayes classifier Intelligent Environments

31. Bayes Classifier • Pros • Optimal • Discrete or numeric attribute values • Naive Bayes easy to compute • Cons • Bayes classifier computationally intractable • Naive Bayes assumption usually violated Intelligent Environments

32. Decision Tree Induction Day Sun M…F Sat Time > 0600 yes no Time < 0700 yes no Locationt … Bedroom Bathroom Intelligent Environments

33. Decision Tree Induction • Algorithm (main loop) • A = best attribute for next node • Assign A as attribute for node • For each value of A, create descendant node • Sort training examples to descendants • If training examples perfectly classified, then Stop, else iterate over descendants Intelligent Environments

34. Decision Tree Induction • Best attribute • Based on information-theoretic concept of entropy • Choose attribute reducing entropy (~uncertainty) from parent to descendant nodes A1 A2 v1 v2 v1 v2 Bathroom (25) Kitchen (25) Bathroom (25) Kitchen (25) Bathroom (50) Kitchen (0) Bathroom (0) Kitchen (50) ? ? B K Intelligent Environments

35. Decision Tree Induction • Pros • Understandable rules • Fast learning and prediction • Cons • Replication problem • Limited rule representation Intelligent Environments

36. Other Prediction Methods • Hidden Markov models • Radial basis functions • Support vector machines • Genetic algorithms • Relational learning Intelligent Environments

37. Prediction Issues • Representation of data and patterns • Relevance of data • Sensor fusion • Amount of data Intelligent Environments

38. Prediction Issues • Evaluation • Accuracy • False positives vs. false negatives • Concept drift • Time-series prediction • Distributed learning Intelligent Environments