Byung Duk Song, Jonghoe Kim, and James R. Morrison*

1. Towards Real Time Scheduling for Persistent UAV Service: A Rolling Horizon MILP Approach, RHTA and the STAH Heuristic ByungDuk Song, Jonghoe Kim, and James R. Morrison* Department of Industrial and Systems Engineering KAIST, South Korea Thursday, May 29, 2014

2. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

3. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

4. Motivation • Increasing demand for commercial small size UAVs • Applications: Patrol, tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief • Limitations: Flight time & mission duration • Long duration, multiple customers and persistent service require - Collection of UAVs, refueling stations, automatic operation system - Methods to orchestrate their operations

5. Motivation: Application Example (1) • Example 1: Tracking a moving ground target Station 1 Station 3 Station 2

6. Motivation: Application Example (2) • Example 2: UAV border patrol Station 1 Station 2

7. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

8. Goal and Ideas • Provide uninterrupted UAV service with continuous appearance of mission requests (or unanticipated system state changes) ■Goal ■Ideas • Persistent UAV operation via service stations • Real time service perspective: • - Incorporate current UAV information (location, fuel level) • - Efficient solving algorithms • Uninterrupted customer service

9. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

10. Mathematical Formulation: Assumptions ■Assumptions 1. Moving target’s path and location at specific times are known. • 2. Current location and fuel levels are known. 3. Recharge time for a UAV depends on the remaining fuel(battery) amount 4. UAV travel speed is constant

11. Mathematical Formulation: Split Job ■Split job: Moving objective path is divided into set of split jobs Service station 2 UAV 2 Job 1 Start UAV 2 5 8 1 9 6 7 4 10 2 UAV 1 End UAV 1 3 ▪ Objective moves - From point (50,250) to (950,350) - From 13:10 to 13:20 Service station 1

12. Mathematical Formulation: Notation ■Notation

13. Mathematical Formulation: Notation ■Notation ■Decision Variables ▪ Xijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the rth flight; 0, otherwise • ▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; • otherwise its value is 0. • ▪ qkr is total fuel (battery) consumption amount for UAV k during its rth flight

14. Mathematical Formulation ■Mathematical formulation Network flow & job assignment

15. Mathematical Formulation Start time constraints Fuel constraints Decision variables

16. Formulation Demonstration (CPLEX) ■Problem description : Problem map Customer 1 Customer 2  Station ▲ UAV S3 U6 U2 U5 U1 S1 U4 U3 S2

17. Formulation Demonstration (CPLEX) ■Problem description : Split job & station & UAV information

18. Formulation Demonstration (CPLEX) ■Result of example 1 - Objective value : 662.570 - CPU time: 5.42 seconds

19. Formulation Demonstration (CPLEX) ■Result of example 2 : Change qk,ini - Objective value : 809.067 - CPU time: 9.17 seconds

20. Formulation Demonstration (CPLEX) ■These examples shows us - Real time perspectives: Current location and fuel level of UAVs - Access to the persistent service - Sharing of multiple stations

21. Formulation Demonstration (CPLEX) ■Computational limit of CPLEX Needs for alternative solution approaches to achieve real time perspective

22. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

23. Solution Approach: Solution Properties ■Analysis on solutions 1) Strict condition: Feasibility issue - Fuel - Split job start time 2) Flexible condition: Solution quality issue - Total moving distance - The number of split jobs which UAV can serve in that flight - Trade off between direct & indirect flight

24. Solution Approach: Solution Properties ■Analysis on solutions 3) Sequential assignment of split jobs

25. Solution Approach: Solution Properties ■ Trade off between direct and indirect flight < Direct flight > 5 8 1 9 6 7 4 10 2 UAV 1 3 Service station 1 • Without replenishment • Short moving distance • Serve less number of split jobs

26. Solution Approach: Solution Properties ■ Trade off between direct and indirect flight < Indirect flight > 5 8 1 9 6 7 4 10 2 UAV 1 3 Service station 1 • Replenishment on the station • Long moving distance • Serve more number of split jobs

27. Solution Approach: STAH ■Based on the analysis of solutions 1) Strict condition 2) Flexible condition Assign UAVs which satisfy strict conditions (Ajk) Contribution value of direct and indirect flight VD(k) = {αw( # of split jobs) –(1-α)(total moving distance) }∙Ajk VI(k) = {αw( # of split jobs) –(1-α)(total moving distance) }∙Ajk Select the flight which provide highest value From customer 1 (earliest customer), split jobs are assigned sequentially

28. Solution Approach: RHTA ■ Receding horizon task assignment algorithm (RHTA) • Classical heuristic (petal algorithm) for vehicle routing problem • Iterative IP model • Generate good quality feasible solution in short time • Classical RHTA was modified: • Eliminate the resource selection components • Modify the equation of replenishment time for a UAV

29. Solution Approach: RHTA ■ Petal : Sequence of split jobs to be served by a UAV • Step 1: Enumerate all feasible petals for each UAV k • Generate petal • Check feasibility of petal • Evaluate of petal value Fuel limitation Time window

30. Solution Approach: RHTA • STEP 2: Solve a single IP to select petal • STEP 3: Assign split job of selected petal to UAVs • STEP 4: Send exhausted UAVs to a service station - if there are remaining jobs on the list, go to STEP 1. • STEP 5: Send any UAVs not located at a station to a station Minimize total travel distance A job can be processed by at most one UAV At least P job should be selected

31. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

32. Results for Various Problem Sizes ■Results • STAH and RHTA obtain very near optimal values. • CPLEX issues an out of memory error. • STAH and RHTA derive feasible solution in a reasonably short time.

33. Real Time Operation with New Service Request Arrival ■During the UAV service of example 1, new service request arrived at T=10. ■Remaining jobs and UAV information at T=11.

34. Real Time Operation with New Service Request Arrival

35. Real Time Operation with New Service Request Arrival T = 18 10 11 12 19 14 15 16 17 13 8 7 4 6 9 5 Customer 3 Station 3 Customer 1 Station 1 Station 2 Customer 2

36. Presentation Overview • Motivation • Goal and ideas • Mathematical formulation • Solution approaches: STAH & RHTA • Numerical examples • Concluding remarks

37. Concluding Remarks • Fleets of UAVs conducting missions over a field of operations can achieve persistence - Supported by shared service stations distributed over the field - Real time operation to handle new arrival of service request • MILP scheduling model - Allows a mobile robot to perform persistent service by sharing base for energy resupply - Maximize service quality by serving 100% service requests by orchestration of components - Demonstration of real time perspectives • Develop a efficient heuristics to overcome computational limit of MILP formulation - STAH - RHTA • Application & numerical examples

38. Back-UP Materials

39. Solution approach: STAH • STEP 1: Arrange customers in the non-decreasing order of (first) split job start time • Let P= {1,2,…,p} be the set of customers and Pt be the set of split jobs in customer index t • Element of Pt also arranged according to the non-decreasing order of split job start time. Set P=1. • STEP 2: From customer P, split jobs are assigned to the UAVs. • Calculate VD(k) and VI(k) • Among VD(k) and VI(k) for every UAV, the biggest function value and corresponding UAV schedule is selected • STEP 3: Update parameters • Current location, fuel level, available time of UAVs • Remaining split job information for customer P is updated ■STAH procedure

40. Solution approach: STAH • STEP 4: Repeat step 2 to 3 until every split job is assigned to UAV in customer P. • STEP 5: Set P=P+1, and repeat step 2,3 and 4. This step is repeated until P=|P|+1. • STEP 6: Send any UAVs not located at a station to a station ■STAH procedure

41. Solution approach: STAH ■Pseudo code (STAH)

42. Solution approach: RHTA ■ Petal : sequence of split jobs to be served by a UAV • Step 1: Enumerate all feasible petals for each UAV k • Generate petal (Limitation of maximum size of petal) • Check feasibility of petal • Evaluate value of petal Fuel limitation Time window

43. Solution approach: RHTA • STEP 2: Solve a single IP for all UAVs to minimize the travel distance • Select petals to be used subject to Minimize total travel distance A job can be processed by at most one UAV At least P job should be selected

44. Solution approach: RHTA • STEP 3: Assign split job to UAV • Assign UAV to a first split job of selected petal • Update system state (UAV location, fuel, available time and job information) • Remove assigned job from remaining job list • STEP 4: Find exhausted UAVs • Send UAVs that do not have any feasible petals to a service station • Update system state • Return to STEP 1 if remaining job list is not empty • STEP 5: Send any UAVs not located at a station to a station

45. Solution approach: RHTA ■Pseudo code (RHTA)