A Hierarchical Goal-Based Formalism and Algorithm for Single -Agent Planning AAMAS ‘12

1. A HierarchicalGoal-BasedFormalismandAlgorithmforSingle-Agent PlanningAAMAS ‘12 Utku Şirin 1560838

2. Outline • Planning and Domain Models • HierarchicalGoal Network (HGN) Planner formalismandproof of itscapabilities • An algorithmfor HGN planning, GoalDecomposition Planner (GDP) • ExperimentalResults • CommentsandConclusions

3. Automated Planning • Whatis automatedplanning? • There is goalandcurrentsituation, aim is toachievethegoalbyexecutingpossibleactions • Currentsituation is definedbystates • Repeatedly; • execute an executableaction, applythechangestothestateandcheckwhetherthegoal is satisfied • How todo theseautomatically, fast, and in lessnumber of steps? • Important ability for computurized agents • Robotic Agents • Game-Playing Agents • Web-service Agents • etc...

4. Domain Models • Planner shouldhave a domain model definingthestates, actionsandtherelationbetweentheactionsandstates • How tobuild domain models? • Hand-craftedplannermodule • Hugedevelopmenteffort ! • Domain-configurableplanner • Utilization of a domain-model file • Most of theusesareHierarchicalTask Network (HTN) Planning • Therearemethodsdividingtasksintosubtasks (will be analyzeddeeper) • Does not focus on goals, but tasks • Justapplythetasksuntilthere is noremainingtasks • Easierwithrespecttohand-craftedplanningmodule • Problem of lacking of taskandgoalcorrespondencemakes it hard totranslateclassicalplanningdomainsinto HTN domains, hencetoprovesoundness • Can we do better? • HierarchicalGoal Network (HGN) Planner • Similarto HTN formalism, but easiertodevelop domain models • Moreflexible • Integrates domain-independentheuristics • Decomposesgoalsratherthantasks • Provably HGN has sameexpressivitypower as HTN, is soundandcomplete

5. Formalism • Classical Planning • Domain D is a finite-statetransitionsystem • S is a set of states, eachstate is a finite set of groundatoms • Ex: onTable(block1), on(block2,block1) • G is thespecification of thegoalstatecomprised of a set of groundatoms • O is a set of operatorswhich is a triple • (head(o), pre(o), eff(o)) • Eachaction is a groundinstance of an operator • An actiona is executable in a statesif s╞pre(a) (s entailspre(a)) • Meaningthat s satisfiesthepreconditions of action a • Afterexecution an action a, thenewstate s’ is • s’ = (s - eff-(a)) ∪eff+(a) • A plan  = <a1, …, an> is executable in s ifeachactionai is executable in thestateproducedby ai-1. • A solutionto a classicalplanning problem P = (D, s0, g) is , ifδ(s0, ) ╞g, whereD is the domain, s0 is theinitialstateand g is a goaldefinition

6. HierarchicalGoal Network (HGN) Planning • Similartoclassicalplanning but havemethodsadditionally • A HGN methodm is a quadruple (head(m), pre(m), sub(m), post(m)) • head(m) andpre(m) same as theones in operatorsforclassicalplanning • sub(m) list of goals <g1, …, gk> whereeachgi is a goalformula • post(m) = gk ; ifsub(m) is non-empty post(m) = pre(m) ; otherwise • Relevance:An action a or a method m is relevantfor a goalformulagifeff(a) or post(m) entails at leastoneliteral in g. • Providessmallersearchspacethan a classicalplanner • A HGN domain is D’ = (D,M) where D is a classicalplanning domain and M is the set of methods

7. Proofs • HGN planning is soundandcomplete. Theseareprovedbymapping HGN planning problem toclassicalplanning problem • Soundness: • HGN planning domain is D = (D’,M), whereD’ is a classicalplanningdomain • Everyactionexecutable in D is alsoexecutable in D’ • Hence, everysolutionto problem P = (D, s0, g) is also a solutiontoP = (D’, s0, g) • Hence, HGN planning is sound. • Completeness • For a pathx in classical domain D, there can be constructed a methodmthatspecifieseachstate in x as a sub-goal in itssub(m). • Then a singleactionwillachieveeachsubgoalcompletingthepath • Henceforeachclassicalplanning problem P = (D, s0, g), there is a HGN planning problem P’ = ((D, M), s0, g)wherePandP’ havesame set of solution

8. Proofs • HGN formalismexpressivitypower is equalto HTN formalism • FromHGN formalismconstruct HTN formalism • Mapsubgoalstosubtaskswithsamepreconditions • <g1, … , gk> mappeddirectlyto <tg1, … , tgk> • In HTN, however, it is neededto define primitivetasksas well. So, define a newprimitivetaskforeachtgihavingsamepreconditionas giandnosubtasks(that’swhy it is primitive, indeed). • From HTN formalismconstruct HGN formalism • Mapsubtaskstosubgoalswithsamepreconditions • <t1, … , tk> mappeddirectlyto <fint1, … , fintk>

9. A little Bit HTN Associatemethodswithnetworks Criticsfordifferenttypes of network

10. Algorithm, Goaldecompositionplanning (GDP)

11. GDP is soundandcomplete • Soundness,if GDP returns a plan, it is a solutionindeed. • Induction on length of thesolutionn • For n = 0, it means s0╞ g • If is a solution of length k < n returnedby GDP • Then’ of length k+1 returnedby GDP is also a solution as line 11 appends a relevantaction/method u tothe plan • Completeness, ifthere is a solution, then GDP willreturn it • Induction on length of thesolutionn • For n = 0, GDP willreturn it as s0╞ g • Assumethere is a solution of length k and GDP returns it • Then GDP returnssolutions of length k+1 as at line 11 GDP appendsrelevantaction/method

12. Domain-independentheuristics • One of themostimportantcontribution of HGN planningformalism • Line 9-13 waschoosingaction/methodsnondeterministically, however, it can be chosenbased on a heruisticvalue • So, line 9-13 will be replaced as below:

13. Domain-independentheuristics • How tocalculateheruisticvalueforeachaction/method: States-Levels Action-Levels First propositionallevel in which p appears in PlannigGraph PLANNING GRAPH

14. Experiments • An HTN planner SHOP2, a classicalplanner FF andthe HGN planner GDP arecompared in fivedifferentdomains • DOMAINs: • LogisticsTransportation Domain: • Thereseveralcities. • At eachcitythereareseveral post-offices • Aim is tomovespecifiednumber of packagestodifferentcities • Intracitytransportation is done viatrucks • Intercitytransportation is done viaairplanes • Trucksandairplanesareunlimited • Blocks-World: • Thereare n-manyblocks in a specifiedconfiguration • Converttheinitialconfigurationtogoalconfigurationbyobeyingthefollowingrules: • Moveoneblock at a time • A blockmay be put on anotherblockortable • Depots: • Combination of LogisticsandBlocks-World domain • Truckshavehoistjustlikethearms of robots in Blocks-World domain • StackingthecratesbecomesBlocks-World domain • Towers of Hanoi: • Therearethreesticks in whichseveraldisksareplaces on it • Disksare put in such a waythateach disk is smallerthanthe disk that it is put on it • Movedisksfromonesticktootherbyobeyingthefollowingrules • Moveone disk at a time • No disk may be put onto a smaller disk • 3-City Routing • Theonlynewlywritten domain, hence it is a weak domain model • Thereare 3 cities • Eachcity has severallocationsandlocationsareconnectedwithroadsarbitrarily in thecities • There is onerandomroadconnected city1 to city3 andonerandomroadconnected city2 to city3 • Aim is togofrom city1 or city3 to city2

15. Results • Logistics Domain Results • Forn = 15, 20, …, 60 packages • GDP-h does not bringmuchoverheadforheuristicfunctioncalculation • FF has strongheuristics

16. Results • Blocks-World Domain Results • Forn = 10, 20, …, 100 blocks • FF has knownproblemswithBlocks-World • GDP-h has heuristicvaluecalcuation time overhead • GDP-h results in a bit smallerplans

17. Results • TheDepots Domain Results • Forn = 10, 20, …, 80 crates • FF cannotsolvemorethan 24 crates • GDP-h heuristicoverhead is significant, alsohavealmostsameplanswiththeotherplanners

18. Results • Towers of Hanoi Domain Results • Forn = 3, …, 14 rings • SHOP2 could not solveproblemsforn > 12and GDP and GDP-h cound not solveproblemsforn > 14 • Both is duetostackoverflow, hencethought as implementationissue, FF did not use a stack • FF has verybadplanningresultswhiletheothershavealmost optimal pathresults

19. Results • 3-City Routing Domain Results • Allpreviousdomainsarestrongandverywelldefineddomains • Thisone is constructed as a weak domain model havingonlyonemethodfor HGN andthreecorrespondingmethodsfor HTN • For n = 10, 20, …, 100 cities • GDP and SHOP2 could not solveexceptforn = 10 • FF maysolvetheproblemsupto n = 60, afterthatpoint it evencould not parsetheproblem file • GDP-h solvedallproblemsquicklyandnearly optimal • Thereasonforthesuccess of the GDP-h is theguidedsearchthankstotheheuristics • As the model is weak, theotherplanners do not haveenoughinformationtoconstraintthesearchspaceand do a lot of backtrackings, however, GDP-h usesheuristicto be abletoguideitssearchandnarrowitssearchspace • As a conclusion, we can say thatifthere is a strong domain model, heuristiccalculationmostprobablywillresult in a overheadandgive not significantlybetterresult; however, ifthe model is weakthancontribution of heuristicfunction is crucial

20. Results • Domain Authoring • Subjectivetodevelopers • Measures as number of lispsymbolsandcomparedfor GDP and SHOP2 planners • GDP almostalwayshavelessnumber of symbols • HTN specifiesmorethanonetasktoachieve a goalformula. Itdefines a decompositiontask, severalprimitivetasksanddeletion-checkconditions, while, GDP onlyneedstospeficifythose as goalsandlettheplannerchoosetheappropriateactionto do with respecttothegoal. • There is a needfordifferent basecasesforeach method in HTN, however, GDP does not needsuch basescases as thesemantic of a goalprovidesto do nothingif a goal is true.

21. CommentsandConclusions • No cross-domain explanationsfortheexperiments. Forexample, why FF is unsuccessfull is not answered. Justtheresultsareshownand it is saidthat GDP is capableenoughtheothers, evenproducesbetterresultsforweak domain models. • Almosteverything is comparedwith HTN but HTN is not explained, at least in principle. Moreover, main difference is not shownalgorithmically. Whatwasdoing HTN andnowwhat is thethingthat HGN is not doing, therebyresultingbetter. Forexample, can weuseheuristics in SHOP2 planner. I guesswe can, andifwe can, it mayalsoproducesimilarresults. • HGN is moreintutivewhencomparingboth, hence, seemsgoodcontributiontotheliterature (since 1974). However, HTN is beingusedmanymanyyears, hencemorecomprehensivecomparison is expected • Sotheonlycontribution of HGN is theeasydevelopment domain models, which is even a subjectivecriteria.

22. References • V. Shivashankar, U. Kuter, D. S. Nau, and R. Alford. A hierarchical goal-based formalism and algorithm for single-agent planning. In Eleventh Internat. Conf. on Autonomous Agentsand Multiagent Systems (AAMAS), 2012 • KutluhanErol, James A. Hendler, and Dana S. Nau. UMCP: A sound and complete procedure for hierarchical task-network planning. In Proceedings of the International Conference on AI Planning & Scheduling (AIPS), pages 249–254, 1994. • J. Hoffmann and B. Nebel. The FF planning system. JAIR,14:253–302, 2001. • D. S. Nau, T.-C. Au, O. Ilghami, U. Kuter, J. W. Murdock,D. Wu, and F. Yaman. SHOP2: An HTN planning system.JAIR, 20:379–404, Dec. 2003. • M. M. Veloso. Learning by analogical reasoning in generalproblem solving. PhD thesis CMU-CS-92-174, CarnegieMellon University, 1992. • F. Bacchus. The AIPS ’00 planning competition. AI Mag.,22(1):47–56, 2001. • M. Fox and D. Long. International planning competition,2002. http://planning.cis.strath.ac.uk/competition. • Hui Li.Technical Report: Relaxed Plan Graph Heuristic Cost Estimation. 2006.

23. AnyCommentsorQuestions ?