OUSD & Air Force Centers of Excellence Air Force Institute of Technology WPAFB Dayton, Ohio

1. Research Thrust 2 Resilient Control and Communication for Large-scale Systems of Autonomous Vehicles Faculty members: Drs. Karimoddini, Jamashidi, Yi, and Kelley OUSD & Air Force Centers of Excellence Air Force Institute of Technology WPAFB Dayton, Ohio January 24, 2019 http://techlav.ncat.edu/ Ali Karimoddini, PhD TECHLAV Center, Deputy Director and leader of Research Thrust 2 ACCESS Laboratory, Director Department of Electrical and Computer Engineering North Carolina A&T State University 1601 E. Market Street/524 McNair Hall Greensboro, NC 27411 Email: akarimod@ncat.edu Website: http://eceserver.ncat.edu/akarimod/ Office: 336-285-3313 Fax: 336-334-7716 The work cannot be used, adapted, copied, or published without the creator’s permission.

2. Teaming of Autonomous Vehicles Systems of Systems: A(large) collection of task-oriented, fairly simple systems that together create a new, more complex networked system which offers more functionality, performance and robustness toward achieving a common (global) goal. Advantages • Several smaller and cheaper agents > one complex agent. • The resulting architecture is robust to the failure of the agents. • The efficiency of the team will be increased. • Distributed load instead of central processing. • Capable of doing different missions such as cooperative search and coverage, surveillance, cooperative mapping, mutual defense, cooperative attack and rendezvous . A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

3. Vulnerabilities … Failures in components Physical attacks Failures in components Failures in communication links Cyber Attacks A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

4. Resilience … • Reliability and maintainability are critical performance attributes for unmanned systems to accomplish their missions and to achieve required operational availability. • Resilience is the ability for an application, system, or subsystem to react to problems in one of its components and still provide the best possible service. Possible strategies in case of failures: • Reconfigure the control structure to maintain the original performance. • Reconfigure the structure to achieve a reduced but still acceptable performance. • Bring the system to a safe mode, e.g. safely land the UAV. • Activate the self-destruction mode. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

5. Thrust 2’s Scope of Work • Thrust 2 (Resilient Control and Communication of Large-scale Autonomous Vehicles (RC2LAV)) will develop systematic methods to enhance the reliability and efficacy of the control structure and the communication backbone for LSASV integrated with human operators in dynamic and uncertain environments such as a battlefield. Sub-thrust 2-1: Developing Fault Tolerant Control Mechanisms for LSASV Sub-thrust 2-2: Developing a Reliable Distributed Communication Network for LSASV Dr. Karimoddini NC A&T Dr. Jamshidi UTSA Dr. Brian Kelley UTSA Dr. Sun Yi NC A&T A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

6. DoD Relevance Physical attacks Failures in components Failures in communication links Cyber Attacks A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

7. Sub-thrust 2-1: Developing Fault Tolerant Control Mechanisms Sub-thrust 2-1 develops a formal framework for fault-tolerant cooperative control of a team of autonomous systems comprised of heterogeneous autonomous vehicles. Fault Detection: If failure has been happened? Fault Isolation: If a failure has been happened, what is the type of failure? Fault Diagnosis Fault Location: If a failure has been happened, where it has been happened? Fault Accommodation: If a failure has been happened, how to handle the failure? Fault Accommodation Fault diagnosis Sensors Controller System A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

8. Developed Diagnosis Approaches We consider different levels of uncertainties: 1- Diagnosis of an Unknown System 2- Diagnosis under unknown initial condition of the system and with unknown past history 3- Diagnosis under partially unknown initial condition of the system and with partially unknown past history . Developed approaches: 1- Active-learning for knowing the system and diagnosing the occurred failures at the same time. 2- Asynchronous diagnosis for a system with unknown initial condition and with unknown past history 3- Semi-Asynchronous diagnosis for a system with partially unknown initial condition of the system and with partially unknown past history. 4- Data-driven diagnosis for a system with unknown model A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

9. Event-driven Diagnosis for an Unknown System Assumption: Failures are abrupt changes in the system, and we may model failures in the system as “events”. Approach • We use the theory of Discrete Event Systems to model the failures. • We develop a “diagnoser” as a diagnosis tool. • In the absence of complete information about the system, we develop an active learning technique to adaptively build-up a diagnosis tool for the system. Occurred Faults Observable behaviors of the system DES behaviors of the system DES Diagnosis Tool Natural Projection Original System P [1] M. M. Karimi, A. Karimoddini, A. P. White and I. W. Bates, “Event-based fault diagnosis for an unknown plant,” 2016 IEEE 55th Conference on Decision and Control (CDC), Las Vegas, NV, 2016, pp. 7216-7221. [2] W. Bates, A. Karimoddini, M. Karimadini, “A Learning-based Approach for Diagnosis and Diagnosability of Initially Unknown Discrete Event Systems,” IEEE Transactions on Automatic Control (Under review). For further details please visit Poster #10, “Health Monitoring of Unknown Autonomous Systems, “ by Wendell Bates, Alejandro White, Ali Karimoddini. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

10. Diagnosis under Unknown/Uncertain Initial Conditions Developed asynchronous and semi-asynchrnuous DES fault diagnosers, which are notrequired to be synchronously initialized with the system under diagnosis (i.e., they can work without requiring the restarting of the system) to automatically, timely, and definitively diagnose (detect, identify and locate) occurred faults. Diagnosis Observations UNKNOWN SYSTEM INITIALSTATE AND CONDITION Fault [1] A. White, A. Karimoddini, “Asynchronous Failure Diagnosis for Discrete Event Systems,” The IEEE American Control Conference, 2017. [2] A. White, A. Karimoddini, “Semi-Asynchronous Failure Diagnosis for Discrete Event Systems,” Proc. of the IEEE International Conference on Systems, Man, and Cybernetics, 2016. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

11. Fault Diagnostics and Prognostics through Data Analytic Approaches Implemented machine learning methods and Deep Learning techniques to discover failures of sensors, actuators, etc. for live data and for a group of vehicles. • Fault created by adding electrical tape to the wheel of the Kobuki A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

12. Sub-thrust 2-2: Reliable Distributed Communication Network Sub-thrust 2-2 designs enhanced efficient and flexible communication network that guarantees the reliable connections among the agents robust against the failures in communication links Failures in communication links Cyber Attacks A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

13. Efficient and flexible communication networks Developing an efficient and flexible hybrid communication technique to: 1- enable neighbor wireless devices in the network to communicate in mmWave band over a direct link rather than going through the network infrastructure, and share the data and tasks among themselves, if they are in Line-of-sight (LoS). 2- Use microwave band and communicate through the network infrastructure in case of blockage. Impacts Challenges • Power Efficiency • Reduce communication delay • Offload cellular traffic from eNB • Spectrum efficiency • Extended coverage area • Mode Selection • Neighbor Discovery • Spectrum Sharing • Interference Management • Power Control • Radio Resource Management [1] N. Namvar, N. Bahadori, and F. Afghah, “Context-Aware D2D Peer Selection for Load Distribution in LTE Networks”, Proc. of 2015 IEEE Asilomar Conference on Signals, Systems, and Computers, pp 464—468, 2015. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

14. Efficient and flexible communication networks Deploying UAVs as Flying Wireless Communication Platforms to extend line-of-sight (LoS) of communication links • Optimal 3-D location of UAVs to maximize the network coverage. • Optimal assignment of the network devices to UAVs and optimal trajectory of UAVs to minimize the energy. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

15. Efficient and flexible communication networks Self Organizing Networks Developing techniques for Self-organizing networks (SON) to increase operational efficiency, and improved planning, deployment and optimization. • Approach: Using Deep Learning techniques to be applied to Self-Organizing Networks to • Optimize features of the network • Improve Quality of Experience (QoE) and Quality of Service (QoS) • Interference mitigation [1] Yerrapragada, Anil Kumar, and Brian Kelley. "An IoT self organizing network for 5G dense network interference alignment," In System of Systems Engineering Conference (SoSE), 2017 12th, pp. 1-6. IEEE, 2017. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

16. Resilient communication networks Delay-Tolerant and Loss-Tolerant Consensus in Networks of Agents Communication between agents often experience delays due to signal transmission, intermittent connectivity and link breakage. • Developed techniques for • Mathematically modeling the delays and loss • Predicting the effects of delays on system performance. • Developed and implemented control techniques to robustly handle the effects of delays [1] M. A. Prince, C. Thomas, S. Yi, “Analysis of the Effects of Communication Delays for Consensus of Networked Multi-Agent Systems” International Journal of Control, Automation and Systems (2017) [2] S. Armah, S. Yi, W. Choi, "Design of feedback control for quadrotors considering signal transmission delays," International Journal of Control, Automation and Systems, vol. 14, pp. 1395-1403, 2016. [3] Stephen K. Armah and Sun Yi, “Analysis of Time Delays in Quadrotor Systems and Design of Control,” in Time Delay Systems: Theory, Numerics, Applications, and Experiments, pp. 299-313, Springer International Publishing. [4] S. Armah, S. Yi, “Altitude Regulation of Quadrotor Types of UAVs Considering Communication Delays,” In Proc. 12th IFAC Workshop on Time Delay Systems, pp. 263-268, 2015. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

17. Resilient communication networks Security and Resource Allocation of IoT networks Due to the large scale of IoT and their interconnected and distributed nature, IoT are vulnerable to security threats compromising reliable, yet efficient service to IoT devices . • Developed IoT control techniques to reliably distribute power over the set of channels. • Used game theory approaches to find the best strategy for combating the jamming attack in IoT networks. [1] K. Yerrapragada and B. Kelley, "Design of K-user massive MIMO networks," 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), New York City, NY, pp. 377-383, 2017. [2] N. Namvar, B. Kelley, “Jamming in the Internet of Things: A Game-Theoretic Perspective”, 2016 IEEE Global Communications Conference, Washington, DC, 2016, pp. 1-6. [3] B. Kelley, G. Parra, D. Akopian, “Cognitive Interference Avoidance in 4th Generation GPS,” In Proc. 2015 10th System of Systems Engineering Conference (SoSE), pp. 410 – 415, 2015. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

18. Highlights of achievements • Failure Diagnosis: • Developed a novel optimal active-learning diagnosis algorithm that can deterministically detect the occurrence of the failure and its type for an unknown autonomous system. • Developed novel asynchronous and synchronous diagnosis techniques that can diagnose fault occurrences without requiring the system under diagnosis to be restarted. • Time-delay in networked controlled systems • Conducted quantitative analysis of effects of signal transmission delays for networked systems via infinite eigen-spectrum analysis. • Developed control strategies which take into account varying delays of networked systems by incorporating estimation of delays into the control loop. • Anti-jamming • A novel centralized anti-jamming strategy for the network controller was developed. • IoT resource allocation • Developed an effective resource allocation strategy for the Unmanned Aerial Vehicles (UAV)-based IoT systems. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

19. Future Plan • Failure Diagnosis: • To develop more robust diagnosis techniques that can handle missing observations. • Develop fault accommodation techniques • Apply the developed diagnosis techniques to different scenarios • Time-delay in networked controlled systems • Considering the delay on cooperative multi-agent systems • Considering time-varying and stochastic delay that may occur on internet-based or long-distance control systems • IoT resource allocation • Develop deep learning techniques for Self Organizing Networks • Develop hybrid mmWave/microwave communication networks A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

20. Publications [14] Allen-Prince, Myrielle, Thomas, Christopher, and Yi, Sun, “Analysis of the Effects of Communication Delays for Consensus of Networked Multi-Agent Systems,” To appear in International Journal of Control, Automation and Systems. [13] A. White, A. Karimoddini, “Asynchronous Failure Diagnosis for Discrete Event Systems,” To appear in Proc of 2017 IEEE American Control Conference (ACC 2017). [12] NimaNamvar, Mostafa Darabi, Walid Saad, and MrouaneDebbah ,“Adaptive Mode Selection in Cognitive Buffer-Aided Full-Duplex Relay Networks with Imperfect Self-Interference Cancellation for Power and Delay Limited IoT devices,” To appear in Proc. of 2017 IEEE International Conference on Communications. [11] Stephen K. Armah and Sun Yi, “Analysis of Time Delays in Quadrotor Systems and Design of Control,” in Time Delay Systems: Theory, Numerics, Applications, and Experiments, pp. 299-313, Springer International Publishing. [10] A. White and A. Karimoddini, “ Semi-asynchronous Fault Diagnosis of Discrete Event Systems,” Proc. of 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3961-3966, 2016. [9] J. Dai, A. Karimoddini, H. Lin, “Achieving Fault-tolerance and Safety of Discrete-event Systems through Learning,” Proc. of the 2016 American Control Conference (ACC), pp. 4835-4840, 2016. [8] M. M. Karimi and A. Karimoddini and A. P. White and I. W. Bates, “Event-based Fault Diagnosis for an Unknown Plant,” Proc. of 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 7216-7221, 2016. [7] NimaNamvar, Walid Saad, NiloofarBahadori, and Brian Kelley, “Jamming in the Internet of Things: A Game-Theoretic Perspective,” Proc. of 2016 IEEE Global Communications Conference (GLOBECOM), 2016. [6] N. Namvar, N. Bahadori, and F. Afghah, “Context-Aware D2D Peer Selection for Load Distribution in LTE Networks”, Proc. of 2015 IEEE Asilomar Conference on Signals, Systems, and Computers, pp 464—468, 2015. [5] S. Armah, S. Yi, “Altitude Regulation of Quadrotor Types of UAVs Considering Communication Delays,” In Proc. 12th IFAC Workshop on Time Delay Systems, pp. 263-268, 2015. [4] T. Okore-Hanson and S. Yi , “Adaptive synchronization of networked multi-agent systems considering transient responses and disturbances,” Proc. of 2016 International Symposium on Flexible Automation (ISFA), pp. 490-496, 2016. [3] S. K. Armah, S. Yi, and W. Choi , “Design of feedback control for quadrotors considering signal transmission delays,” Journal of International Journal of Control, Automation and Systems, Vol 14, No 6, pp. 1395-1403, 2016. [2] Stephen K. Armah and Sun Yi, “Analysis of Time Delays in Quadrotor Systems and Design of Control,” in Time Delay Systems: Theory, Numerics, Applications, and Experiments, pp. 299-313, Springer International Publishing. [1] Modeling and Analyzing the Effects of Delays-Consensus of Networks of Multi-Agent Systems (MS thesis of Myrielle Allen-Prince). A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

21. Publications [15] M. A. Prince, C. Thomas, S. Yi, “Analysis of the Effects of Communication Delays for Consensus of Networked Multi-Agent Systems,” International Journal of Control, Automation and Systems (2017) [16] Yerrapragada, Anil Kumar, and Brian Kelley. "An IoT self organizing network for 5G dense network interference alignment," In System of Systems Engineering Conference (SoSE), 2017 12th, pp. 1-6. IEEE, 2017. [17] K. Yerrapragada and B. Kelley, "Design of K-user massive MIMO networks," 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), New York City, NY, pp. 377-383, 2017. [18] B. Kelley, G. Parra, D. Akopian, “Cognitive Interference Avoidance in 4th Generation GPS,” In Proc. 2015 10th System of Systems Engineering Conference (SoSE), pp. 410 – 415, 2015. A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center

22. Acknowledgement • TECHLAV Faculty Members Mo Jamshidi Ali Karimoddini Brian Kelley Sun Yi Graduated Students • Graduate Students NiloofarBahadori Myrielle Allen-Prince NimaNamvar Taylor Eisman Christopher Thomas Wendell Bates Shubham Sarpal Alejandro White • Undergraduate Students Temesgen Gebregziabher Daniel Tobias Alan Kruger Tyisheam Jackson William Gray Kenneth Lindsay • And special thanks to DoD, OSD, Air Force Research Laboratory, and ARL for supporting this research A. KarimoddiniThrust 2: Resilient Control and Communication TECHLAV Center