AI and Software/System Engineering

1. AI and Software/System Engineering Jingyue Li Associate Professor Department of Computer Science NTNU IPIT Annual meeting at University of Michigan, June 2019

2. My background • Education • Master (Artificial Intelligence) • Ph.D. and Postdoc (Software Engineering) • Industrial experience • IBM China (Software Engineer 2001-2002) • DNV GL (Principal Researcher 2010-2016) Associate Professor at NTNU from 2016, teaching • Software security • Advanced software engineering

3. My research focuses • Engineering AI • Safety verification of AI-based system • Defend against AI-based security attack • Defend against AI-based privacy compromise • AI for Engineering • Use AI to improve traffic management and safety • Use AI to improve system safety

4. To address safety challenges of using AI • Researchers are working on* • Assuring robustness of NN (Neural Network) • Improving failure resilience of NN • Measuring and ensuring test completeness of NN • Assuring safety prosperities of NN based CPS • Improving interpretability of NN • Safety certification (e.g., IEC 61508) • Completeness, Correctness, Repeatability, Precisely defined testing configuration * Jin Zhang and Jingyue Li: “Testing and verification of neural network based safety-critical control software: A systematic literature review,” submitted to a journal.

5. AI for improving security Fujitsu AI increases accuracy of malware intrusion detection* *https://phys.org/news/2017-10-fujitsu-ai-accuracy-malware-intrusion.html

6. Malicious use of AI for password guessing * * Hitaj, Briland, et al. "Passgan: A deep learning approach for password guessing." arXiv preprint arXiv:1709.00440 (2017)

7. AI-based malware

8. The AI-Based Cyber Threat Landscape planning Intrusion Execution Stage Exploitation Command and control Actions Delivery Reconnaissance Weaponization Installation AI-targeted AI-aided AI-concealed AI-automated AI-evolved AI-multilayered AI-massive E.g., Data breach, DDos E.g., remain undetectable E.g., activate existing destructive behavior or establish multiple paths of attack E.g., self propagating E.g., generate adjusted attack payload E.g., new way to exploit E.g., learn the targets’ standard behavior Objective * Nektaria Kaloudi and Jingyue Li: “The AI-Based Cyber Threat Landscape: A Survey,” submitted to a journal.

9. AI to improve privacy • AI actually increases the potential for consumer privacy by reducing the number of humans who see their personal information*. * https://iapp.org/news/a/ai-offers-opportunity-to-increase-privacy-for-users/

10. Privacy vs. Security

11. Machine Learning (ML) and anonymization Processed using ML ML-based privacy attack Raw personal data Anonymized data De-anonymized personal data * Lee et al. “Blind De-anonymization Attacks using Social Networks”, 2018 Jingyue Li et. al, “A survey on ML-enabled anonymization and de-anonymization”, on-going.

12. My study focuses • Engineering AI • Safety verification of AI-based system • Defend against AI-based security attack • Defend against AI-based privacy compromise • AI for Engineering • Use AI to improve traffic management and safety • Use AI to improve system safety

13. Analyzing data from sensing cars for improving traffic management

14. Predictive maintenance of machinery • Failure or service prediction • Anomaly detection

15. Other initiatives • Automatic identification of vulnerable code • Automatic fixing of vulnerable code

16. Thanks! Questions? Software Engineering and System engineering AI