Fundamental concepts and terminology of Verification and Validation

1. Fundamental concepts and terminology of Verification and Validation • Verification is the process that checks whether mathematical model was implemented accurately enough. • Validation is the process that checks that simulation is close enough to reality. • This terminology is not universally accepted. • 1979 figure from Oberkampf and Roy

2. Paper Helicopter model qualification • When dropped, paper helicopter starts autorotating, which slows down its fall, like a helicopter when it loses power. • Has very simple math model: Newton’s second law – F=ma • F=Weight-drag • Differential equation .

3. Verification • In order to verify the computational model and its implementation (e.g. software) we need to test it against analytical solutions or special cases where the solution of the differential equations is known. • For the paper helicopter we will first demonstrate with analytical solution, and then with a numerical solution. .

4. Analytical solution of paper helicopter velocity .

5. Verifying the analytical solution • We start by checking that at t=0 the velocity is zero at that at infinity the velocity goes to Vs. • Then we substitute solution into the differential equation to check that it is satisfied:

6. Top hat question • What conditions can we apply to check if the expression for dV/dt is correct? • We know its value at t=0. • We know its value at t=infinity. • Both. • Neither.

7. Numerical solution • Numerical solution may allow us to relax some of our assumptions, such as constant drag coefficient (may be function of speed). • The simplest way to integrate a differential equation is with Euler explicit algorithm with constant time step • Can get also height without the challenge of analytical integration • Now need to implement in software. • Will use Matlaband verify against analytical solution.

8. Matlab code function [v,va,h,t]=paper_copter(v0,c,g,T,n) % Calculates trajectory of paper helicopter both % analytically and by Euler explicit integration. dt=T/n; vs=sqrt(g/c); t=linspace(0,T,n+1); va=vs*(1-exp(-2*vs*c*t))./(1+exp(-2*vs*c*t)); v(1)=v0;h(1)=0; for i=1:n v(i+1)=(g-c*v(i)^2)*dt+v(i); h(i+1)=v(i)*dt+h(i); end • g=32.2;c=1;T=2;v0=0.;

9. But… • If we check Matlab code for negative (upward) initial speeds. • g=32.2;c=1;T=0.5; v0=-5.7; • [v,va,h,t]=paper_copter(v0,c,g,T,10); v • -5.7000 -5.7145 -5.7373 -5.7731 -5.8295 -5.9187 -6.0602 -6.2866 -6.6526 -7.2554 -8.2775 • It is flying into space! • What error did I stumble on?

10. Code verification • Includes numerical algorithm testing and software quality assurance. • However, there are several errors that may create false impression that code is deficient.

11. Model validation • Three stages according to Oberkampf and Trucano

12. Calibration • Model often includes parameters that need to be estimated from measurements. • For paper helicopter we can measure air density and area. • We estimate drag coefficient from final speed measurement. • We may introduce calibration parameter of fake initial condition to account for period when it falls without spinning.

13. Distinguishing between alternative models • Consider the possibility that a model for paper helicopter where drag is proportional to speed instead of square of speed is suggested. • That is, Newton’s second law is • Solution

14. Top Hat question • Which of the following is a valid difference between linear and quadratic model • Linear model does not suffer from sign problem when helicopter flies up. • Linear model approaches steady state faster. • In linear model mass has larger effect on steady state speed.

15. Equivalent calibration • We will assume that we measure the steady state speed, and use it for calibration. • For quadratic model • For linear model • With errors not be easy to distinguish

16. Anti-optimization for model discrimination • Principle: Large errors are easier on debugging than small ones. • Example: Comparing finite element and finite strip calculations of buckling loads for stiffened panels I observed 0.3% difference. • I knew that the difference should be at most 100 times smaller. • I did not have a chance of getting any attention from software developers for this problem. • I played with problem parameters until the difference grew to 100%. • It became clear that finite-element solution was wrong, and it took developer one day to find the error.

17. Anti-optimization for composite failure criteria • Van Wamelen A., Haftka, R.T., and Johnson, E.R., "Optimal Design of Laminated Specimens to Evaluate Competing Composite Failure Criteria," Proceedings American Society for Composites, 8th Technical Conference, Technomic Publishing Company, 1993, pp.1045-1055 • Designed laminate with ply angles that caused two failure criteria that usually give close results to lead to factor of 2 difference. • Test validated Tsai-Wu failure criterion, against Hart-Smith criterion. • However, developer of competing criterion claimed boundary failure.

18. Anti-optimization for drag models • Maximize ratio of quadratic and linear speeds What can we do to improve experimental conditions for measuring difference?