Nonlinear Regression

Nonlinear Regression Daniel BaurETH Zurich, Institut für Chemie- und BioingenieurwissenschaftenETH Hönggerberg / HCI F128 – ZürichE-Mail: daniel.baur@chem.ethz.chhttp://www.morbidelli-group.ethz.ch/education/index Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Example: Puromycin • Puromycin is an antibiotic used in bio-research to select cells modified by genetic engineering • The Michaelis-Menten model for enzyme kinetics relates the initial velocity of an enzymatic reaction to the substrate concentration xby the equation • At high concentrations, the velocity is essentially constant (θ1), while at low concentrations it is a linear function of the concentration Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

The model: Puromycin Kinetics Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Model Linearization • The initial model reads • This can be rearranged to • Renaming the variables yields a linear model Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Regression Line b1 = 0.0051072 b2 = 0.00024722 Linearized Model: Fit Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Regression from Linearized Model Regression from linearized model q1 = 195.8 q2 = 0.048407 Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Nonlinear Regression • The objective function of non-linear regression readswhere y is the vector of responses, x is the vector of observations, θ is the vector of parameters and f is the nonlinear model function • In the 2-parameter case, we can plot S(θ) as a function of the parameter values to look for a minimum Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Minimum Estimated value of q from linearization Objective Function S(θ) Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Minimization of S(θ) • If we linearize the model with respect to the parameters, we getwhere J(θ0) is the Jacobian matrix of f with respect to θ, evaluated at θ0 • In this case, the residuals are • If we now define the objective function in terms of the residuals, we can apply the Gauss-Newton method Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Gauss-Newton Method applied to S(θ) Convergence path of Gauss-Newton Method (q1)opt = 212.66 (q2)opt = 0.064091 Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Nonlinear Regression Nonlinear Regression Regression from linearized model Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Matlab Nonlinear Regression Routines • There are several options for doing nonlinear regression: • lsqcurvefit: Best for quickly fitting models to data; Not much statistical output; Provides more numerical information and options; Can use bounded parameter ranges • nlinfit: General nonlinear regression routine; Provides some statistical output • NonLinearModel: Most useful for nonlinear regression; All statistical and numerical tools in one place; Works analogously to LinearModel Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Using the Nonlinear Regression Routines • First we need to create a function that takes as inputs a vector b, the model parameters, and a vector (or matrix) x, the data points of the observed variables, and returns as output a vector (or matrix) y, the model responses • Then we use one of the nonlinear regression routines: Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Plotting the Fitted Curves • Plotting is as simple as calling the cost function or the predict routine of the NonLinearModel Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Tukey-Anscombe Plots • Plot of residuals vs. fitted values Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Normal Probability Plot Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Estimation of the Confidence Intervals Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Exercise 1 • The product of a chemical reaction contains 50% chlorine at the time of production. The amount of chlorine subsequently drops over time. A model is proposed to predict the chlorine concentration as a function of time:where X is the fraction of chlorine in the product, t is the time and α and β are the model parameters. Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Assignment 1 • Find online the data file chlorine.dat which contains the measurements and some code to load the data. • Plot the chlorine concentration as a function of time.Would a linear model seem feasible? • Determine suitable initial values for the nonlinear regression by linearizing the model and using linear regression to estimate the parameters. • Use either regressor a LinearModelto perform the linear regression. • Since the dependence is exponential, simply use a log-model to linearize the model: Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Assignment 1 (Continued) • Write a model function of the form • functionXCl = chlorinemodel(par, t) • where par(1) will be α and par(2) will be β • Fit a nonlinear model to the data by using one of the functions given on slide 12. Plot the model with the data, and report the parameters. • If you use NonLinearModel, you can use as input either a datasetor the measurements as X, y pair. Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Exercise 2 • An experiment was conducted to evaluate the effect of two different methionin nutrition supplements (sourceA and sourceB) on the growth of newborn turkeys. The turkeys were separated into 10 experimental units with 15 birds each. For each supplement, the amount fed was varied between 0.04% and 0.44% of the total food. The target variable, the mean body weight of the units, is expressed as a function of the dosage of the two supplements, according to the following non-linear modelwhere Y is the mean weight, and xA and xB are the dosages of the two supplements, respectively. Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Exercise 2 (Continued) • The model assumes that the intercept θ1 and the horizontal asymptote θ1 + θ2 are the same for both supplements, and that only the growth rates θ3*θ4and θ3for supplements A and B, respectively, are different. The question is whether one of the supplements is superior, i.e. whether θ4 is significantly different from unity. Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Assignment 2 • Find online the data file turkeyweight.dat and some code to load the data into Matlab. • Find suitable initial values for the parameters by linearizing the model. • If you cannot solve this use θ1 = 638, θ2 = 175, θ3 = -5 and θ4 = 1. • Write a model function of the form • function y = turkeymodel(theta, x) • Perform the nonlinear regression, using nlinfitor NonLinearModel. • Are the growth rates for the two supplements significantly different, i.e. is θ4 significantly different from unity? • Use either nlparcior coefCIto estimate the confidence intervals. Daniel Baur / Numerical Methods for Chemical Engineers / Nonlinear Regression

Nonlinear Regression