Model-based Programming of Cooperating Robots

Model-based Programming of Cooperating Robots Brian C. Williams, Jonathan Kennell, I-hsiang Shu, and Raj Krishnan Artificial Intelligence and Space Systems Labs Massachusetts Institute of Technology in Collaboration with Maria Fox and Jon How

Outline • Model-based Programming of Mobile Vehicles • Example: Cooperative Mars Rovers • Kirk Model-based Executive for Mobile Robots • Demo: UAV Squadron Simulation – direct control • Coaching Heterogeneous Teams • Example: Forest Fire Rescue • Kirk with Generative Activity Planner • Demo: UAV Squadron Simulation – indirect control • Integration with MILP Path Planning • Demo: Forest Fire Rescue on ATRV trucks • Demo: UAV Testbed

Example:Programming Cooperating Rovers OEP Wireless sensor networks

RMPL control program Strategy Selection Strategy macro decomposition TPN Planner Visibility Graph Kirk Model-based Execution System Overview Mission Developer Strategy Macro Library Mission Controller schedulable plan

Example Enroute Activity: Enroute Corridor 2 Rendezvous Rescue Area Corridor 1

RMPL for Group-Enroute Group-Enroute()[l,u] = { choose { do { Group-Fly-Path(PATH1_1,PATH1_2,PATH1_3,RE_POS)[l*90%,u*90%]; } maintaining PATH1_OK, do { Group-Fly-Path(PATH2_1,PATH2_2,PATH2_3,RE_POS)[l*90%,u*90%]; } maintaining PATH2_OK }; { Group-Transmit(OPS,ARRIVED)[0,2], do { Group-Wait(HOLD1,HOLD2)[0,u*10%] } watching PROCEED } at RE_POS }

RMPL control program Strategy Selection Strategy macro decomposition TPN Planner Visibility Graph Kirk Model-based Execution System Overview Mission Developer Strategy Macro Library Mission Controller schedulable plan

Enroute Activity Encoded as a Temporal Plan Network Enroute[450,540] 1 2 [0, 0] [0, 0] Group Traverse Group Wait 4 [405, 486] 5 9 [0, 54] 10 [0, 0] [0, 0] [0, 0] [0, 0] Ask( PATH1 = OK) Ask( EXPLORE = OK) Science Target 8 13 3 [0, 0] [0, 0] Group Traverse Group Transmit [0, ] [0, 0] 6 [405, 486] 7 11 [0, 2] 12 Ask( PATH2 = OK) Activity (or sub-activity) Duration (temporal constraint) Conditional node Symbolic constraint (Ask,Tell)

[500,800] s e Tell(PATH1=OK) Tell(PROCEED) [10,10] [0,] 14 15 16 17 [450,450] [200,200] Instantiated Enroute Activity • Add environmental constraints Group-Enroute [0,] [0,] [450,540] 1 2 Group Traverse Group Wait Ask(PATH1=OK) Ask(PROCEED) 4 9 5 10 [405,486] [0,54] Science Target 3 8 13 Group Traverse Group Transmit Ask(PATH2=OK) [0,] 6 7 11 12 [405,486] [0,2] Activity (or sub-activity) Duration (temporal constraint) Conditional node Symbolic constraint (Ask,Tell) External constraints

Generates Schedulable Plan Group-Enroute [500,800] s e [0,] [0,] [450,540] 1 2 Group Traverse Group Wait Ask(PATH1=OK) Ask(PROCEED) 4 9 5 10 [405,486] [0,54] Science Target 3 8 13 Group Traverse Group Transmit Ask(PATH2=OK) [0,] 6 7 11 12 [405,486] [0,2] Tell(PATH1=OK) Tell(PROCEED) [10,10] [0,] 14 15 16 17 [450,450] [200,200] • To Plan, . . . perform the following hierarchically: • Trace trajectories • Check schedulability • Satisfy and protect asks

Improve Temporal Consistency Checking Algorithm by using Incremental Methods • Observation: • Frequent temporal consistency checks allow for improved planning speed by using a fast incremental temporal consistency algorithm, ITC. • Both when a plan breaks and during incremental candidate plan generation, new candidate plans are similar in structure to previous candidate plans. • Solution: • Reuse the work of previously computed candidate plans, and update only those nodes that need to be updated. • Similar to common incremental algorithms such as Incremental A*, Dynamic A*, and Truth Maintanence System (TMS). • Evalution: • Data shows that using the ITC algorithm improves temporal consistency checks by about an order of magnitude for test cases involving cooperative air vehicles.

Comparison of the Number of Queue Insertions for Temporal Consistency Checking Algorithms Comparison of Temporal Consistency Checking Algorithm Runtime Performance Improvements Temporal Consistency Algorithm (April ’03) Queue Insertions ITC Algorithm (current) Number of UAVs Temporal Consistency Algorithm (April ’03) ITC Algorithm (current) Algorithm Runtime Number of UAVs • UAVs performs 5 total activities, in which 2 randomly selected targets are chosen.

Demo 1: Detailed Guidance Rearm Station 15 Targets 4long range SAM sites circle indicates range 9 Combat UAV Blue Force Headquarters Refuel Station • Mission Specification: • Start with 9 combat UAVs at Headquarters. • UAVs go to station to rearm. • Each UAV then concurrently: • Goes to its assigned target and attacks. • If short-range SAMs active, attack last (risk is high), • Else, attack SAMs first (before they become active). • Goes to station for refueling. • Confirms the success of its attack. • All UAVs then return to Headquarters. • Demonstrates: • Complex coordination of activities. • Consistent option selection • Simultaneous roadmap path planning • Dynamic scheduling

Example: Coaching Heterogeneous Teams • Search and Rescue • Ocean Exploration OEP A dozen vehicles is too many to micro manage • Act as a coach: • Specify evolution of state and location.

Forest Fire Rescue • Goal: retrieve family from fire. • Rescue cannot take place until the local fire is suppressed. • Retrofit one rescue vehicle for fire suppression Fire Line Forest Rescue Point Ambulance Fire

Game Plan • Goal: retrieve family from fire. • Rescue cannot take place until local fire suppressed. • Retrofit one vehicle for fire suppression Fire Line Forest Rescue Point Ambulance Fire Phase 1

Game Plan • Goal: retrieve family from fire. • Rescue cannot take place until local fire suppressed. • Retrofit one vehicle for fire suppression Fire Line Forest Rescue Point Ambulance Fire Phase 2

RMPL control program Strategy macro decomposition environment and action data Strategy Selection TPN Planner Kirk Model-based Execution System Overview Mission Developer Strategy Macro Library Mission Controller state configuration goals Activity Planning • Strategy Selection determines the optimal rules / strategies to accomplish mission goals. • Activity Planning figures out how to achieve mission goals within strategic framework using available low-level actions. Operators, Tactics, Scenario Model Visibility Graph Generative Activity Planner Human / ComputerInterface schedulable planwith rationale MILP Path-Planning

RMPL Control Program • (defclass rescue-team (execute ()(sequence(parallel [l1,u1] (tell-start(at uav1 Ambulance)) (tell-start(at uav2 Ambulance)) (ask-end(suppressed Fire)) )(parallel [l2,u2] (tell-start(at family RescuePoint)) (ask-end(rescued family)) (ask-end(at uav1 Ambulance)) (ask-end(at uav2 Ambulance)) )) )) Initial State Phase 1 IntermediateState Phase 2 Goal State

Environment Model • Terrain Map • Object instantiations: • UAV uav1 • UAV uav2 • RESCU-READY uav1 • RESCUE-READY uav2 • IN-DISTRESS family • LOCATION Ambulance • LOCATION Fire • LOCATION RescuePoint

Vehicle Specifications • Vehicle linearized dynamics • Vehicle primitive operators: • Fly(V,A,B) • move UAV “V” from location “A” to location “B” • Refit(V) • Prepare UAV “V” to drop fire retardant • Drop(V,A) • Drop fire retardant at location “A” with UAV “V” • Rescue(V,P,A) • Rescue people “P” in distress with UAV “V” at location “A”

RMPL control program Strategy macro decomposition environment and action data Strategy Selection TPN Planner Activity Planning Visibility Graph Generative Activity Planner Kirk Model-based Execution System Overview Mission Developer Strategy Macro Library Mission Controller state configuration goals • Strategy Selection determines the optimal rules / strategies to accomplish mission goals. • Activity Planning figures out how to achieve mission goals within strategic framework using available low-level actions. Operators, Tactics, Scenario Model Human / ComputerInterface schedulable planwith rationale MILP Path-Planning

Translate to Planning Problem with Atomic Operators Use Atomic Generative Planner (GraphPlan – Blum & Furst) To Generate Operators and Precedence Extract Temporal Plan and Check Schedulability Kirk Constructs Vehicle Activity Plan Using a Generative Temporal Planner • Approach: • Encode Goal Plan using an LPGP-style encoding • Prototype using LPGP [Fox/Long, CP03] Mission Goal State Plan Generative Temporal Planner Vehicle Operator Definitions Vehicle Activity Plan

[10,20] [20,+INF] [0,100] Generated Activity Plan Refit-Inv [10,+INF] Refit-End Refit-Start Fly-Inv [20,+INF] Fly-Inv [20,+INF] Fly-Start Fly-Start Fire Suppress-End Fly-Inv [20,+INF] Suppress-Start Suppress-Inv [10,20] Fly-End CP-Start CP-Inv-1 [0,100] CP-Inv-1 [0,100] CP-Inv-1 [0,100] CP-Inv-1 [0,100] CP-Inv-1 [0,100] CP-Inv-1 [0,100] CP-Inv-1 [0,100] CP-Midpoint Kirk extracts a least commitment plan and generates a rationale

Demo 2: Strategic Guidance Rearm Station 2 Targets 4 SAM site (obstacle) circle indicates range Blue Force Headquarters 1 Combat UAV Refuel Station 2 Reconnaissance UAV • Mission Specification (as states): • available vehicles at Rearm, Refuel & HQ. • All three UAVs must be fully fueled. • The two targets must be destroyed and confirmed,and the UAVs back at HQ. • Demonstrates: • High-level state commanding • Generative activity planning • with Vehicle assignment

RMPL control program Strategy macro decomposition environment and action data Strategy Selection TPN Planner Activity Planning Visibility Graph Generative Activity Planner Human / ComputerInterface MILP Path-Planning Kirk Model-based Execution System Overview Mission Developer Strategy Macro Library Mission Controller state configuration goals • Strategy Selection determines the optimal rules / strategies to accomplish mission goals. • Activity Planning figures out how to achieve mission goals within strategic framework using available low-level actions. Operators, Tactics, Scenario Model schedulable planwith rationale

3 UAV’s cross paths Straight line paths produce collision “Roundabout” is the best maneuver Vehicles pass at limits of avoidance regions Requires coordination Solution emerges from the optimization problem Aircraft Avoidance Example Work by Jon How et al., MIT

Trajectory Planning withObstacle Avoidance • Representation of obstacles (A) as variables in a CSP • Selection of values a assign a value to each clause vi in V, and results in a feasible region that the vehicle can be (B) • Other selections result in infeasibilities (C) • Goal of trajectory planning is to develop a path across the terrain that avoids all obstacles by resolving all clauses in the CSP

Trajectory Planning withObstacle Avoidance • Search through the space of possible resolutions to the obstacles will result in infeasibilities at certain nodes • Search is guided by leveraging Conflict-Directed A*, which identifies a kernel of a conflict and uses it to skip over other infeasible states • Fast execution is dependent on a way of identifying and using (extracting) conflicts quickly

Mountain Scenario Tactic: Red to suppress Fire before blue comes within its range Safe point UAV(with Fire Suppressant) UAV(rescue) Family Fire

Mountain Scenario Small Fire Both take direct route Blue clear during Fire suppression

Mountain Scenario (continued)

Mountain Scenario Larger Fire radius Still both on direct route Blue in range before fire suppression

Mountain Scenario Larger Fire radius Blue goes via staging point Blue clear when fire suppressed

Mountain Scenario (continued)

Experiment • Fire radius large • SAM radius small Demonstration of combined Capabilities

Cloud Cap Transmitter UAV Testbed Avionics Unplanned Events Groundstation Trajectory Optimization Plan Processor Cloud Cap Interface 900 Mhz Datalink • Autopilots perform waypoint control • Planner interacts exactly as with trucks Mission State Plan Decisions Operator

Dynamic Responses to Environment A B

Summary – Cooperative Robot Scenarios • Combined activity / path-planning • Should solve unified problem optimally • Solution: integration with visibility graph, RRT, MILP • Intra-team dependencies • Achieving mission goals depends on coordinated control, meeting deadlines, etc. • Solution: temporally-flexible TPN plan representation • Heterogeneous teams • Need to allow different operators / dynamics for different types of vehicles • Solution: RMPL • Overlapping vehicle capabilities • Need to solve assignment problem efficiently • Need to support contingencies in case of vehicle loss • Solution: fast planning algorithms, reactive planning with programmer-specified contingencies

Model-based Programming of Cooperating Robots

Model-based Programming of Cooperating Robots

Presentation Transcript

Programming Epson Robots

[Low-level] Programming of NXT Robots

Market-Based Coordination of Recharging Robots

Behavior-Based Robots

Computer Programming with Robots

A Programming model for failure-prone, Collaborative robots

Cooperating Function of Cells

Logic Programming Based Model Transformations

Standards-Based Programming Model

Vision-Based Motion Control of Robots

Model-Based Programming of Intelligent Embedded Systems

Model-based Programming of Cooperating Explorers

An Approach to Model-Based Control of Frictionally Constrained Robots

Remote Educational Programming Of Robots ( REPOR )

Cooperating Out of Poverty

Emotion-Based Control of Cooperating Heterogeneous Mobile Robots

Of programming GUIs, robots and aircraft pilots

SENSOR BASED CONTROL OF AUTONOMOUS ROBOTS

Translation Validation of Compilers for Model-based Programming

Programming Robots using Java

Programming Model