1 / 34

COMP 650: POMDP’s real life applications

COMP 650: POMDP’s real life applications. Rahul Kumar Department of Computer Science Rice University April 18, 2013. Long-term user intention prediction for wheelchair navigation using POMDP.

sitara
Download Presentation

COMP 650: POMDP’s real life applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. COMP 650: POMDP’s real life applications • Rahul Kumar • Department of Computer Science • Rice University • April 18, 2013

  2. Long-term user intention prediction for wheelchair navigation using POMDP • References: Taha et al POMDP-based long-term user intention prediction for wheelchair navigation. • *image taken from http://robotzeitgeist.com/tag/dementia

  3. Outline • Motivation • Introduction • POMDP quick review • Problem Specification / formulation • On Line assistance • Experimental results • Conclusions

  4. Motivation Why to make wheel chairs smart ? • Growing number of aging population. • Increase in accidents or other calamities. • Terrible diseases which affect motor control

  5. Reactive Wheelchairs • Reactive refers to systems that do not use representation of environment. • Most Popular among Intention recognition wheel chairs. • Rely on local or temporal information collected online. • Systems with limited power or processing power use this technique. • Examples: Rolland-III, NavChair etc.

  6. POMDP - 1 • General framework for sequential decision making where states are hidden and actions are stochastic. • Widely used in assistive applications.

  7. POMDP -2 • S – set of states • A – set of action • Z – set of observations • T – conditional transition probabilities S x A x S -> [0,1] • Z – conditional observation probabilities A x S x Z -> [0,1] • R: A x S -> real number

  8. POMDP agent overview Observation Action Environment StateEstimator BeliefState Policy

  9. POMDP Generation For efficient POMDP system , we need to have proper • State Space • Transition States • Observation States

  10. POMDP model generation architecture

  11. State Space • Spatial States : Wheelchair location = {s1,s2,s3,… } • Destination states : Places of Interest = {d1,d2,..} • Joint representation of both of them = {s1d1,s2d1,…}

  12. State extraction and representation

  13. Transition model • Transition model specifies the probability of transition from one state to another given when a certain action is executed. • Actions= global navigation commands = {North, South, East, West, Stop } • Observation = Joystick movements = { Up, Down, Right, Left, NoInput} • Directly calculated from the map topology.

  14. Observation model • We use training data from particular user. • In indoor settings, wheelchair user usually performs repetitive tasks. • For example, A task can be going from living room to kitchen etc.

  15. Reward function • -1 for each action • +100 for an action that leads to Destination.

  16. On-line navigation

  17. Experimental result -1 • Artificial data was generated based on the activity of user in the environment. • Zmdp software package was used. Zmdp package has several heuristic search algorithm for POMDPs and MDPs. • Known starting points but unknown destinations. • 100% success in predicting destination.

  18. Experiment result -2

  19. Experiment result -3

  20. Conclusion • Employing POMDP for long term user intention prediction for wheel chair navigation. • No behavioral selection like other papers.

  21. Future work • Enhance the capabilities and the intelligence of the system through automated activity monitoring and task extraction.

  22. POMDP Hands * Image taken from http://matanyahorowitz.com/index.php

  23. Overview • Motivation • Approach/Big Picture • Example/ Intution • Model Construction • Results

  24. Motivation If you know all shapes and positions exactly, you can generate a trajectory that will work *Slide taken from Hsiao etal.

  25. Problem at hand • How to decide on configuration of object when robot have to manipulate an object!

  26. Approach/Big Picture • Partition Space : Identify and separate regions where we will have similar properties. • Reducing uncertainty in configuration by taking actions which acts as “funnels” i.e. mapping large sets of initial states to smaller set of resulting states. • We will work with set of guarded complaint motion. These actions acts as funnels.

  27. Example Partial policy graph for robot

  28. Abstract model construction • Action space: Two guarded complaint move commands for each degree of freedom. • Transition probability: Sample large number of triplets from given initial states. • Observation probability : Contact sensors have some uncertainty in determining contact. • Reward : 15 for reaching the goal, -50 for lifting in wrong configuration, -1 for each motion, -5 for being in unstable states or boundary states

  29. Solving POMDP

  30. Experiment • Similar to previous problem except that block is stepped • High fidelity simulation : 92% success, average reward: -1.59 • Fixed policy : 81% success, average reward= -10.632

  31. Video demonstration

  32. Future work • To address problem with shape uncertainty. • To handle interaction with other objects,

  33. References • Shio et al Grasping POMDP’s

  34. THANK YOU!

More Related