CSE 551 Computational Methods 2019/2020 Fall Chapter 1 Introduction

1. CSE 551 Computational Methods 2019/2020 Fall Chapter 1 Introduction

2. Outline Introduction Computational Problems Approximations in Scientific Computations Broad Classification of Computational methods Course Related Issues Preliminaries – Nested Multiplication Preliminaries – Taylor Series

3. References • Based on • M. TG. Heath, Scientific Computing: An Introduction, 2ed ed, Mc Graw Hill. • Chapter 1: Introduction • S. . Chapra, Numerical Methods for Engineers: with Software and Programming Applications, Mc Craw Hill. • Introduction to Pert I

4. Introduction • numerical analysis – scienfific computing • design and analysis of algorithms • for solving mathematical problems with aritmetic operations • in many fields – • science and engineering • recently social sciences • quantities continuous v.s. discrete • functions and equations – underlying variables • time, distance, velocity, temperature, presure, density,stress and like

5. most problems in continuous math • derivatives, integration, nonlinearities • cannot be solved exactly – in finite number of steps • iterative process – converges to a solution • the answer is approximately correct • close enough to the desired result

6. finding rapidly convergent iterative algorithms • assesing accuricy of rssulting approximation • if rapid • some problems with finite algorithms – systems of linear eq. – better with iterative methods

7. Approximations • effects of approxmimations • many solution teckhniques • approximations – of verious types • even the aritmetic • digital computer cannot represent all real numbers exactly • numerical algorithms • efficient, reliable and accurate

8. NonComputer Methods • analytical or exact methods: • limited class of problems • linear models • simple geometries • low dimensionality • useful and excelent inside to the behavior of systems • limited practical value • most real problems • nonlinearities • complex shapes and processes

9. Graphical solutions: • characterize behavior • plots or chats • complex problems but results are not very precise • low dimensional – three or fewer • e.g., phase diagrams in thermodynamics • much effort and energy on solution technique • rather than problem formuolation and interpretation

10. Computational Problems • many problems scientific computing from • science or engineering • social sciences, business – computational social scinece • ultimate aim • understand some natural, social phenomena • design a device • computational simulation: • representation or emulation of a physical, social system or a process using computers • greatly enhence scientific understanding by allowing the investigation of situations • difficult or impossible – ttheoretical, observational or experimental means alone

11. Examples • In astrophysics – behavior of two collding black hodes • too complicated to determine theoretically – analytical methods • impossible to observe directly • dublicate in lab • to simulate it computationally requires only • an approximate mathematical representation – Einstein’s equations of general relativity • an algorithm to solve these equations numerically • a sufficiently powerful computer

12. Examples (cont.) • investigate normal situations with less cost and time • Engineering design – large number of design options are tried • quickly, inexpensively, safely • than with treditional “bulid-and-test” methods using physical prototypes v.s. virtual prototyping • e.g., improving automobile safety – crash testing • less expensive and dangerous on a computer • space of design parameter explored more throughly • drug design – computational biochemistry • social policy programms – impossible to meke experiments on society

13. Problem Solving Process in Computational Simulation • Develop a methematical model • expressed with some equations some type – equation based modeling EBM v.s. agent-based-modeling ABM • representing the physical phenomena or the system • Develop algorthms to solve the equations numerically • Implement the algorithms on a computer • Run the algorithms on the computer • Reprsent the computed results – comprehensible form – graphical visuliztion • Interpret and validate the computed results

14. Step 1 – mathematical modeling • domain knowldge particular scientific or engineering disiplines • applied mathematics • Step 2,3 – designing, analysing , implementging numercal algorthms – main subject of scientific computing • Principles and methods of scientific computing • studied fairly broad level in generality

15. but keep in mind • specific sources of a problem and the uses • original problem formulation may affect • accracy of numerical results which affects • interpretation and validation of these reslults

16. Well-posedness v.s. Ill-posssedness • a mathematical problem is well-posed • if a solution exists unique and • depends continuously on the problem data • a small change in data does not cause an abrubt disproportionate change in the solution • in numerical computations • such perturbations are usually inavitable • well-possedness - highly desirable • but not always atchievable

17. An Example • e.g., infering the internal structure of a system from external observations • in tomography or seismology – mathematical problems – ill-posed • distincly different internal configurations • may have indistinguishable external apperances

18. Sensitivity • even a problem is well-possed • the solution may be sensitive to perturbations to perturbations in data or parameters • develop quantitative measures of sensitivity • local and global sensitivity • robustness to alternative assumptions or processes • Sensitivity of algorthms stable algorithms

19. General Strategy • replace a dificult problem with an easier one • same or closly related solutin • E.g.: • infinite dimensional spaces with finite dimensional spaces • infinite processes with finite processes • integrals or infinite series with finite sums • derivatives with finite differences • differential equations with difference equations (algebric equations) • nonlinear problems with linear problems

20. Replacements (cont.) • high order systems with low order systems • complicated functions with simple functions • polynomials • general matrices with matrices with a simpler form

21. Example • to solve a system of nunlinear differential equations • first, replace with system of nonlinear algebric equations – difference equations • then, replace the nonlinear system with a linear one • then, replace the natrix of the linear system with a special form • solution is easy to compute • at each step – verify that • within some tolerance of the true solution

22. an alternative problem(s) easier to solve • a transformation of the given problem to the alternative one • preserves the solution in some sense • much effort • identify class of problems with simple solutions • solution preserving transformations into these classes

23. ideally – solution of transformed problem is identical to the original problem • not always possible – approximate • accuracy arbitarily good – additional work and storage • primary concern • estimating accuracy of such an approximate solution • establishing convergence to the true solution in the limit

24. Approximations in Scientific Computing • Sources of approximation • some before the computation begins • Modeling: • some features of the system under study may be ommited or simplified • friction, viscosity, air resistance • Emprical measurments: • lab instruments – finite precision • accuricy – further limited • small sample size • reading - random noice or systematic bias

25. e.g., • even most careful measurments of physical constants – Newton’s gravitgational constant, Plank’s constant – eight or nine significant decimal digits • most lab measures less accurate than that • Previous computations: • input data – from previous computationla step • may be approximate • beyond our control • determining accuricy expected from a computation

26. approximations we do have some influence • systematic approximations during computation • Trancation or discretization: • some features of a mathematical model may be simplfied or ommited • e.g., replacing derivatives with finite differences or • using only a finite number of terms in an infinite series

27. Rounding: • in computations • by hand, with a calculator or a computer • representations of real numbers and • aritmetic operations upon them • ultimately limited to finite amount of presicion • generally inexact

28. accuricy of final results of a computation • reflect combination of any or all – approximations • resulting perturbations may be amplified • nature of the problem being solved and/or • the algorithm being used • error analysis: • study - effects of such approximations on • the accuracy and the stability • numberical algorithms

29. Example: Approximations • surface area of the Earch A = 4r2, • number of approximations: • Earch as a sphere – idealization of its true shape • value of radius  6370 km – combination • empirical measurment- previous computation • The value of  ifinite process – trancated at some point • numerical values of • the input data and results of aritmetic operations • rounded in a computer

30. Broad Classification of Computational Methods • Roots of nonlinear equations • Systems of linear algebric equations • Optimization • Curve Fitting • Interpolation • Integration • Ordinary Differential Equations • Partial Differential Equations • Monte Carlo methods

31. Roots of nonlinear equation(s) • Roots of nonlinear equation(s) • finding value(s) of a variable that satisfy a single or a set of nonlinear equations f(x) = 0 • problems in engineering design context • mass, energy, force balance, Newton’s laws of motion • analytical solutions • quadratic equations ax2+bx+c=0 • cubic equations – more complex • Abel (1802-1829) proved that no formula existrs for fifth-order poynomials

32. even such a simple function f(x) = e-x – x = 0 • canot be solved analytically • graphical techniques • plot the function and examine the root(s) visually • rough estimate of roots – lack precision • trial and error: • repeat • guessing a value of x, evaluating whether f(x) • until f(x) is sufficiently close to 0 • inefficient and inadequate formany realstic problems

33. systematic strategies with computers • simple and efficient • bracketing methods: • start with a guesses that brackets or contains the root • and systematically reduce the width of the bracket • bisection and false position • open methods • trial and error but no guess of a bracket • computationally more efficient but nay not work • e.g., Newton-Rapson and extensions • graphical methods provide inside • Roots of polynomials

34. Systems of linear algebric equations: • Find values of a vectorial variable that satisfy a set of linear equations Ax = b in matrix form • many problems in verious disiplines • msthematical modeling of large systgms of interconnected elements • such as structures, electrical circuits and fluid networks • other areas of numerical methods • curve fitting and differential equations

35. direct methods: find the solution in fixed or finite number of computatgional steps • Gaussian elimination • iterative methods: produces a sequence of approximate answers • designed to converge ever closer to the true solution under the proper conditions • Direct meethod – exact result if computations were carried out in an exact aritmetic • the effect of numerical round-off may be significant for large linear systems

36. for iterative methods: • question of convergence • Do the succesive approximate answers approch to the ture solution? • if so, how quicly? • how should the decision be made to terminate the process?

37. Optimization • Optimization: • determining value(s) of a scaler or vector variable that corresponds to the “best”: or optimal value of a function – maximum or minimum • engineering design, production planning curev fitting • constraint or unconstraint • linear or nonlinear programming • integer, continuous, mixed integer • dynamic programming • stochastic programming • optimal control problems – determining the best function to optimize a functional

38. Curve Fitting • fitting curves to data points – regression vs interpolation • regression: significant degree of error associated with data • regression: model input – output relation • uses • prediction and understanding (inference) • linear v.s. nonlinear functional forms • output variable – continuous or categoriacal (classification) • machine learning/ data mining • neural networks, support vector machines • parametric v.s. nonparametric

39. Interpolation • objective is determine intermediate values between relatively error free data • usual case for tabulated information • The strategy: • fit a curve directly through the data points • use the curve to predict the intermediate values

40. Integration • geometric interpreation: area under a curve • many other applications • single or multiple integration • finding center of mass • cumulative probability distributions • solution of differential equations

41. Ordinary Differential Equations • many physical laws - rate of change of some variables • e.g., population growth rate, force lows • initial value and boundary value problems • linear constant coefficient – analtical solutions • linear/nonlinear • single or systems of equations – computation of eigenvalues and eigenvectors • deterministic/stochastic

42. Partial Differential Equations • characterize systems - the behavior of a physical quantity is expressed as its rate of change with respect to two or more independent variables • e.g., • steady state distribution of temperature on a heated plate (two spatial dimensions) • time variable temperature of a heated rod (time and one spatioal dimension) • two different approaches to solve numerically • finite difference methods: • finite-element methods:

43. Course Related Issukes • Prerequisites • Requirements • Outline of the Course • Web Page

44. Preequisites for the course • Mathematics courses • calculus • linear algebra • diffeential equations • Programming • knowledge of a programming language

45. Requirements • Homework problems • can be done in any programming languse • Midterm • Final • Project • Project and homework can be done in at most groups of two

46. Outline of the Curse • Introduction • Error Analysis • Solution of Nonlinear Equations • Interpolation and Polynomial Approximation • Numerical Differentiation and Integration • Linear Systems of Equations– Direct Methods • Linear Algebra – Iterative Methods • Computing Eignevalues and Eigenvectors • Curve Fitting

47. Outline of the Curse (cont.) • Ordinary Differential Equations –Initial Value Problems • Ordinary Differential Equations - Boundary Value Problems • Numerical Solutions of Partial Differential Equations

48. Web Page • Web page for CSE 551 • misprivate.boun.edu.tr/badur/CSE551 • You can find • References • Lecture slides • Anouncements • Homework Problems • Project Guidelines

49. Preliminaries • Nested Multiplication • Review of Taylor Series

50. Nested Multiplication • some remarks • on evaluating a polynomial efficiently • on rounding and chopping real numbers • To evaluate the polynomial p(x) = a0 + a1x + a2x2 +· · ·+an-1xn-1 + anxn • group the terms in a nested multiplication: p(x) = a0 + x(a1 + x(a2 +· · ·+ x(an-1 + x(an)) · · ·))