Science Informed Learning

1. Science Informed Learning Goal: The 2-4 most important AI research opportunities described in 1-2 slides: The research opportunity Specific research goals (including expected capabilities in 5, 10, & 15 years) DOE science and engineering impact potential (tied to Day 1 outcomes)

2. Physics Constrained Machine Learning Science Informed Learning Submitted by Sebastian De Pascuale Utilize equations governing physical observables as an additional constraint on neural network training, discriminating against unrealistic solutions Can identify unknown variables determining physical evolution Can produce synthetic datasets to design device experiments Current experiments in fusion energy have limited diagnostic capabilities or can only be run for a brief amount of time Operation of a commercial fusion energy plant will require vast real-time collection of data and actionable control DOE can combine state-of-the-art fusion experiments with world class HPC resources to construct and validate neural network architectures

3. Foundational Limits of AI/ML Crosscut Breakout: Science Informed Learning Session Submitted by Nagi Rao, raons@ornl.gov ML/AI methods are being applied to increasingly complex problems in various DOE areas. But, they do not solve all problems, as some are simply too complex for machines - e.g. detection of zero-day computer viruses, resilient codes to arbitrary hardware failures Identification: classes of problems that are undecidable (non-Turing computable), inexpressible (non-Tarski) and unlearnable (infinite Vapnik-Chervonenkis dimension) Characterization: Analytical and mathematical characterizations of limits and their interpretation within application/domain context • This important area is largely not addressed by existing capabilities and efforts • Results from this area will help: • avoid wasting of valuable resources in trying to solve unsolvable problems • develop domain specializations that lead to solvable AI/ML formulations • DOE has applied math and computer science programs under ASCR both of which are vital to addressing this area

4. How do we define Science Informed Learning? • Learning consistent with theory, existing models, or other physical constraints • Enforce known correlations between inputs and outputs • Enforce known correlations among outputs • What about scientific data? • Are there challenges specific to scientific data (e.g., too little or too much, noise, indirect measurement, missing data, labeled or unlabeled, provenance)? • Enforce constraints before data reduction • Preserve cross-correlation between datasets if we reduce data at the edge • What about scientific literature? • NLP of existing literature to constrain search spaces; multi-modal learning

5. What we heard yesterday... • Combinatorially large search space; physical constraints reduce space • Materials, Chemistry, and Nanoscience | Biological and Life Sciences | Energy Generation and Distribution • Surrogate models for scale bridging consistent with known physics • Materials, Chemistry, and Nanoscience | Climate | Biological and Life Sciences |Fusion Energy | Fundamental physics | Mobility and Transportation (combustion) | Advanced manufacturing • Emergent constraints to constrain future projections based on contemporary observations of states or variability • Climate • Physics-informed learning to ensure ML models follow physics for control of experimental systems • Scientific user facilities | Fusion energy

6. Why do we need science-informed ML? • Reduce parametric/combinatorial search space • Simplify data-driven models (lower-dimensional structure) • Discover unknown correlations that could be masked by “known physics” signal • Meaningful distance on manifold (proper metrics) • Anomaly detection? • When are physics constraints a necessity or just an aesthetic? • What else…?

7. Are there concerns about Scientific Informed Learning? • What is the information content of data after removing known physics? • What bias is introduced by assuming known physics? • Are there fundamental limits on ML (undecidable, inexpressible, unlearnable) that must be respected, i.e., well-posedness for ML? • What else…?

8. What ML methods might work? • Optimization constraints / cost functions • Network architecture design • Reinforcement learning • Augmenting experimental data with synthetic data • Determining latent spaces • Transfer learning • Hyperparameter tuning • What else…?

9. When/Where to enforce science-based constraints? • During the data capture stage • During the learning phase • During the inference phase • Cost savings/model (what is the cost vs. gain) for enforcing the constraints • Where else…?

10. Time frame • What can be done now? • For what problems is lack of physical constraints a limiting issue? • What are the critical mileposts in 5, 10, and 15 years?

11. Possible AI research topics • Reliable, efficient methods to include physical constraints during data reduction/cleaning/etc • Reliable, efficient methods to include physical constraints during training