Predicting the mechanical response of oligocrystals using deep convolutional neural networks

1. Predicting the mechanical response of oligocrystals using deep convolutional neural networks Ari Frankel, Reese Jones, Coleman Alleman, Jeremy Templeton

2. Overview • Additive manufacturing and material variability • Simulation framework • Prediction of elastic modulus • Validation • Prediction of distribution • Prediction of stress-strain curve 3D convolutional layer, k=1, filters=16, ReLU, ‘same’ 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Max pooling, k=2 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Max pooling, k=4 Dense, 20, ReLU Dense, 10, ReLU

3. Additive Manufacturing and Material Variability

4. AM • 3D printing is increasing in popularity • Highly customizable parts/complex geometries • Melted/sintered material “printed” into desired shape: plastic or metal Makerbot/Wiki All3DP

5. AM • Problem: high variability in mechanical response of printed materials • May not be reliable to use these parts in high-risk applications From Rizzi, Jones, Templeton, Ostien, and Boyce, CMAME 2019

6. Sources of material variability • Variations in manufacturing… • Temperature • Annealing time • Printer balance, geometry, loading • Sintering process • Laser power, width • … Result in variations at the mesoscopic level • Imperfections in geometry • Porosity • Grain size and morphology • Impurities/inclusions • To what extent can we predict changes in mechanical behavior due to meso/microscopic level variations? What distribution in properties can we expect? Can we infer geometric features from a given set of mechanical properties?

7. Stress-strain curve Ultimate tensile stress And fracture Yield stress Young’s modulus From wikipedia

8. At the microstructural level EBSD data of stainless steel (courtesy of Brad Boyce) Porosity in a printed tensile specimen

9. Simulation Framework

10. Synthetic microstructural data Grain sizes drawn from log-normal distribution 31 grain topologies created Many textures per topology 20x20x20 mesh

11. Synthetic microstructural data • Each grain is cubic crystal structure • Sample texture orientations using Dakota LHS • Apply time-dependent Dirichlet BC to x-face • Simulates a material under tension in the x-direction • Up to 6% strain • Using LCM-Albany for finite element simulations • Data set: • 31 geometries, 30 textures per geometry • For 10 of the geometries, sample an additional 570 textures

12. Synthetic microstructural data Higher yield strain, Higher final stress, Higher initial stiffness Same polycrystal structure Different grain orientations Colored by x-component of orientation vector Lower yield strain, Lower final stress, Lower initial stiffness

13. Predicting the Young’s modulus

14. Predicting the Young’s modulus • Variability driven entirely by grain topology and textures • Classic homogenization theory • Voigt average: assume strain state in each grain is identical, UPPER BOUND • Reuss average: assume stress state in each grain is identical, LOWER BOUND • Hill average: average of Voigt and Reuss • Similar theories for crystal plasticity (Taylor, Sachs)

15. Predicting the Young’s modulus True modulus Voigt Average Predicted modulus Reuss Average True modulus

16. Predicting the Young’s modulus • Neither Voigt/Reuss do very well, and Hill is totally ad-hoc • Can we use data-driven models? • Develop convolutional neural network to learn Young’s modulus directly from the oligocrystal

17. Representing the texture • Textures were sampled randomly on SO(3) • 3 parameters to represent the orientation of crystal lattice • But the crystals have cubic symmetry! • Redundancies in representation • For any point in SO(3), there are 23 other points with exact same physics • Unnecessary parameter space will slow down machine learning • Use cubic symmetry operations to collapse input parameter space SO(3) to smaller volume

18. Representing the texture Elastic modulus of a single crystal as a function of orientation The ”fundamental zone” under cubic symmetry

19. Convolutional Neural Network 20x20x20 mesh, 3 channels input 3D convolutional layer, k=1, filters=16, ReLU, ‘same’ 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Max pooling, k=2 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Max pooling, k=4 Dense, 20, ReLU Dense, 10, ReLU ~10k parameters

20. Training • Hold each of 20 grain topologies out • Train CNN on remaining data • Keras-Tensorflow • Adam optimizer • Learning rate = 0.0005 • 300 epochs • Standardize inputs and outputs • ~6600 training examples, trained on GPU • Test each CNN on the held out grain topology (each texture)

21. Predicting the Young’s modulus: Uncertainty propagation 97.4% correlation between NN and truth 1/N1/2 dependence of variance of E

22. Predicting the stress-strain curve • Our interest extends far beyond elastic deformation • Can we predict other interesting features? Yield, saturation of plastic flow? • Use a convolutional neural network + recurrent neural network to predict time series • (Limit to the first 10 time steps of the deformation)

23. CNN-LSTM LSTM 10, tanh/ReLU Time Repeat vector Dense(time), 10, ReLU 3D convolutional layer, k=1, filters=16, ReLU, ‘same’ 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Max pooling, k=2 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Time vector is constant for all realizations. Only thing changing is the CNN output. LSTM output is a sequence of vectors. 3D convolutional layer, k=2, filters=16, ReLU, ‘same’ Max pooling, k=4 Dense, 20, ReLU

24. Training • Use data augmentation: 8 different ways of holding the geometry that give the same stress-strain curve • Limit prediction to the first 10 time steps (just past yield) • Hold out 5 grain topologies (and their rotations) • Keras-Tensorflow • Adam optimizer • Learning rate = 0.0005 • 30 epochs (begins to overfit after this) • Standardizing the inputs and outputs (of the mean curve, using a constant scale) • 50k training examples • Trained with GPU • Evaluate CNN-LSTM on the held out grain topologies

25. CNN-LSTM: validation/propagation (De-)correlation of NN prediction from ground truth as a function of strain Comparison of NN prediction of distribution versus other classic homogenization theories

26. Conclusions and future directions • ”Predicting the mechanical response of oligocrystals with deep learning”, Frankel, Jones, Alleman, Templeton, arXiv:1901.10669 • Application of data-driven modeling to capture functional behavior of stress-strain curve as function of oligocrystal directly • Uncertainty propagation of oligocrystal samples to predict distribution of responses • Use of material physics to reduce parameter space for more efficient learning • Applications: multiscale simulation and structure-property relations • Big questions: • What did the NN learn that classic homogenization theory missed? • Can we interpret the findings of the trained NN? • Is it trustworthy? • How to perform inference of microstructural features from macroscopic behavior?