General Least Squares Techniques for Curve Fitting

1. GG313 Lecture 16 General Least Squares 10/13/05

2. Mid-Term next Tuesday Go over homework

3. Least Square techniques for matching a model to data are useful for matching curves other than straight lines. • Note that you never never never adjust your data to match a model! • When we use our data to obtain a model, we are inverse modeling. In forward modeling, you generate “fake data” to compare with real data. • You can try to fit any function to your data, from a straight line to an exponential function, polynomial, cosine, whatever. Commonly you will fit a straight line to your data, subtract the resulting line from your data, and thus remove a trend. Sea level data are often dominated by tidal fluctuations, made up of a combination of sine waves that vary in amplitude with the solar and lunar cycles, these can be removed to reveal smaller changes.

4. Gravity fluctuations are generated by changes in the density of the earth beneath you that can be modeled by changing the densities of polygonal bodies in the sub-surface. • Earthquake locations are obtained by assuming travel times of seismic waves through the earth, which in tern depend on the velocity of those waves in the rocks. By adjusting the seismic wave velocities in various parts of the earth, the travel time errors can be reduced, and improvements can be made in our knowledge of the earth’s interior. • Each of these examples, and many others involve the solution of many simultaneous equations, and they are all applications of least squares and linear algebra.

5. What we are trying to do is fashion a model, often called a set of basis functions, that will minimize the errors obtained when we subtract the model results from our observational data. The model parameters are always constrained by our prior knowledge of the physics and/or geological constraints. For example, we cannot use negative densities in gravity problems, or negative velocities in seismic travel times. We almost always want to utilize fewer parameters in our model than we have data points, otherwise our model is as complex as our data, and the purpose of the model is to simplify and provide insight. It’s possible to obtain a perfect fit to your data if your model has as many parameters as you have data parameters.

6. We start with a set of basis functions that we expect will provide insight into the problem we are trying to solve - sine waves for tidal data, velocities for seismic waves, densities for gravity, that are assigned to particular locations or times depending on the problem. Each of these functions will have a coefficient representing an unknown value that we multiply by the function. We want to solve for the values of these coefficients that minimize the errors in our comparison between the model and the data. Again, small errors do NOT validate the model, they only mean that the model is consistent with the data. Consider y(x)=a1f1(x)+a2f2(x)+…+amfm(x) In gravity, y would be the expected gravity at a point x that

7. is made up of the contributions of many polygonal bodies of various shapes and distances from x. We wish to fit this model to a set of n data points, such as gravity values obtained along the x profile, where we have more data points than model coefficients (n>m). We do this by minimizing the error, E: (3.113) We minimize E in the least squares sense by looking for the values of ai where: (3.115)

8. For the general term in eqn. 3.113, (3.118) We can rearrange the terms of the equations so that we get one equation for each j coefficient, yielding a set of normal equations: (3.120)

9. In matrix notation: (3.122) Or very simply: With solution: (3.123) (3.124) Yielding the least squares solution to out model parameters.

10. In Paul’s notes, he presents another solution to the above case, but it has the same result. He also presented weighted mean square solutions. You should know that this derivation is in the notes, but we won’t go over it.

11. Review for the mid-term exam