DETERMINING THE ORDINARY DIFFERENTIAL EQUATION FROM NOISY DATA

1. DETERMINING THE ORDINARY DIFFERENTIAL EQUATION FROM NOISY DATA P. Venkataraman

2. Todays Presentation Problem Definition Solution Steps Recursive Bezier Filter The Inverse Problem Example with Low Noise Example with High Noise Things Learned Conclusion Future Work

3. Problem Definition GivenNoisy Data (an ordered set of x and y values) • Find : • the ordinary differential equation (ODE) that describes the data • the boundary conditions associated with the data • Some Assumptions: • ODE is determined from smooth data • Initial and final data are uncertain • Solution is compared by numerically integrating the ODE

4. Solution Steps The solution is established through the following steps • Step 1: Smooth the data using a recursive Bezier Filter. This establishes a smooth function of the data and also all its derivatives. • Step 2: Choose the generic form of the ODE. The specific ODE is determined through two kinds of unknowns. (i) exponents of the derivatives (integers) and (ii) the coefficients of the terms of the ODE (real numbers) • Use Discrete optimization establish the unknowns of the ODE through minimization. The objective function is the sum of the error in the residuals of ODE over the data points in the interior. • Integrate the ODE and compare the trajectory to the smooth data.

5. Recursive Bezier Filter The Bezier Filter (fit a smooth Bezier function to the data) • 1. Choose a starting order for the Bezier function that will fit the data. • 2. Find the Bezier coefficients using Least Squared Error (LSE) • 3. Find the sum of the absolute error (LAS). • 4. Increment the order by 1 and repeating Step 2, and calculate the change in the LAS. • 5. If the LAS decreases, then the order is increased by one, and the processing is returned to Step 1. If it increases then this current order and coefficients will determine the best Bezier function. The Recursive Bezier Filter • 1. Apply the Bezier Filter(with order at least 8 for nonlinear data) • 2. Decrease the order of the Bezier function by 1 and fit the current filtered data(not the original noisy data) by applying the Bezier filter. • 3. Return to Step 2 (for a prescribed number of applications- 4 or 5)

6. Recursive Bezier Filter Fifth recursion First recursion Noisy data and Bezier filter Effect on first derivatives Effect on second derivatives Effect on third derivatives

7. The Inverse Problem We assume the data represents a third order, homogenous, linear, constant coefficient differential equation Unknowns: (i) e3, e2, e1, e0 (integers) (ii) a3, a2, a1, a0 (real numbers) The unknowns are established by minimizing the sum of the residuals through optimization Objective function : Because of trivial solution we use discrete optimization Algorithm used: Basic Simulated Annealing (Bohachevsky) with new design vector generated heuristically MATLAB 2009, memory and time were not an issue

8. The Inverse Problem Objective function : In computing the objective function all of the derivatives are known from Step 1 Even if the ODE is established through optimization, it must be integrated to compare with the fitted smooth data. Often the residuals are not small enough to ensure that integration is not necessary. The integration uses state space methods. • For data with large noise the residuals within a clipped region is used to identify the ODE. • This ODE is integrated backwards to establish the BC at the initial point. • It is integrated forward to establish the BC at the final point.

9. Example with Low Noise The noisy data was created using random number A known third order linear ODE is used to generate the noisy data No. of points: 31 Order of curve: 5 LSE : 6.1398e-004 LAS: 0.1226 No clipped region

10. Example with Low Noise Solution: e3 = 1; e2 = 1; e1 = 1; e0 = 1 a3 = 1; a2 = 0.75; a1 = 0.75; a0 = 0.5 Both ODEs are integrated using exact BC’s. All reruns of this example did not converge to a particular form of the differential equation even though their integration determined graphically similar and close trajectories.

11. Example with High Noise The noisy data was created using random number The same third order linear ODE is used to generate the noisy data No. of points: 31 Order of curve: 4 LSE : 0.1222 LAS: 1.657 Clipped region (ignore 5 data points from either end)

12. Example with High Noise Solution: e3 = 1; e2 = 1; e1 = 1; e0 = 1 a3 = 0.5; a2 = 0.25; a1 = 0.5; a0 = 0.25 Both ODEs are integrated using exact BC’s.

13. Example with High Noise Solution with New BC: Calculated Initial Conditions : Calculated Final Conditions : ODEs are integrated using proper BC’s.

14. Things Learned Some Existential Questions (not answered in the work) Is there a unique, continuous, and smooth representation of the underlying data? – number of smooth representations is exhaustive! Does there exist smooth and continuous derivatives of the uncertain data? – there are smooth derivatives based on the continuous representation of the data which is not unique Is there a unique ODE corresponding to data?–even if the derivatives are not unique! Is there unique trajectory if the boundary conditions are inexact? How can distance between two different ODE’s be defined? (different coefficients but trajectory is close!) Additional Questions • What is the least number of data points to solve the inverse problem? • Do we need to use all of the data points to establish the solution? –Justification of the clipped region! (ODE represents the entire domain) • Are there additional constraints on the layout of the data?

15. Conclusions • A two-step approach is used to identify the ODE from noisy • Step 1: Identify the best Bezier function that fits the data. This also determines the derivatives of the function. • Step 2: Find the exponents and coefficients of an appropriate generic ODE by minimizing the sum of the residuals of the ODE 2. Discrete optimization is used to avoid trivial solution. 3. Simulated Annealing is used to identify the solution 4. The discovered ODE is not unique 5. No change in the procedure is anticipated for nonlinear ODE or coupled ODE’s. once a suitable nonlinear generic form is chosen.

16. Future Work • A hybrid approach using symbolic and numeric computation has been recently developed for the Inverse problem for ODE. It has the following features: • Determining the Bezier functions • The Bezier description is symbolic/analytic • The coefficients are determined using unconstrained optimization • Establishing the coefficients of the ODE • The form of the ODE is assumed and the exponents are considered known • The coefficients are symbolic • The coefficients are established using unconstrained minimization based on the minimization of the residuals • Reestablishing the coefficients of the ODE • The coefficients are numeric and are determined through unconstrained optimization • The objective is the error between the original data and the numerical integration of the ODE over points in the trajectory • The previous known solution is used to start the unconstrained optimization

17. Questions ?