90 likes | 207 Views
Two Models of Evaluating Probabilistic Planning. IPPC (Probabilistic Planning Competition ) How often did yo u reach the goal under the given time constraints FF-HOP FF- Replan. Evaluate on the quality of the policy Converging to optimal policy faster LRTDP mGPT Kolobov’s approach.
E N D
Two Models of Evaluating Probabilistic Planning • IPPC (Probabilistic Planning Competition) • How often did you reach the goal under the given time constraints • FF-HOP • FF-Replan • Evaluate on the quality of the policy • Converging to optimal policy faster • LRTDP • mGPT • Kolobov’s approach
Heuristics for Stochastic Planning • Heuristics come from relaxation • We can relax along two separate dimensions: • Relax –ve interactions • Consider +ve interactions alone using relaxed planning graphs • Relax uncertainty • Consider determinizations • Or a combination of both!
Determinizations • Most-likely outcome determinization • Inadmissible • e.g. if only path to goal relies on less likely outcome of an action • All outcomes determinization • Admissible, but not very informed • e.g. Very unlikely action leads you straight to goal
Solving Determinizations • If we relax –ve interactions • Then compute relaxed plan • Admissible if optimal relaxed plan is computed • Inadmissible otherwise • If we keep –ve interactions • Then use a deterministic planner (e.g. FF/LPG) • Inadmissible unless the underlying planner is optimal
Dimensions of Relaxation 3 4 Negative Interactions Increasing consideration 1 2 Uncertainty Relaxed Plan Heuristic 1 Reducing Uncertainty Bound the number of stochastic outcomes Stochastic “width” McLUG 2 FF/LPG 3 4 Limited width stochastic planning?
Dimensions of Relaxation Uncertainty -ve interactions
Expressiveness v. Cost Node Expansions v. Heuristic Computation Cost Limited width stochastic planning FF McLUG Nodes Expanded FF-Replan Computation Cost h = 0 FFR FF
Reducing Heuristic Computation Cost • Exploit overlapping structure of heuristics for different states • E.g. SAG idea for McLUG • E.g. Triangle tables idea for plans (c.f. Kolobov)
Triangle Table Memoization • Use triangle tables / memoization C B A A B C If the above problem is solved, then we don’t need to call FF again for the below: B A A B