Adaptive, Robust Regression for Functional and Image Data in Functional Mixed Model Framework

1. Adaptive, Robust Regression for Functional and Image Data in Functional Mixed Model Framework Hongxiao Zhu, Philip J. Brown, Jeffrey S. Morris

2. Outline • Introduction • Functional Mixed Models/Isomorphic Modeling • Gaussian Wavelet-Based Functional Mixed Models • Robust Functional Regression • General Model • Specific Implementation • Outlier Diagnostics • Application • Simulation Study • Conclusions

3. Functional/Quantitative Image Data • Functional Data: Ideal units of observation: curves • Fine Grid: • Benefits: observe nearly entire curve • Difficulties: high dimension, maybe less than ideal smoothness • Examples: actigraphy data, EEG data, ERP data, time series, data on spatial grid, various genomics and proteomics data • Course/Random Grid: • Benefits: smaller sample size, can assume functions smooth • Difficulties: don’t observe entire curve, must borrow strength across curves to recapture entire functional profile • Different methods for these two types? Here fine grid • Quantitative Image Data: functional data on fine grid • Scanned images: intensities interpretable quantitatively • Data set involves multiple images; interested in combining information across images to make inferences about populations • Examples: fMRI data, 2D gel electrophoresis proteomics data, PET data, Spatio-temporal data

4. Functional Mixed Model (FMM) • FMM: extension of LMM to function/object data • Guo (2002), Morris and Carroll (2006), plus others • Observe sample of Nfunctions, Yi(t),i=1, …,N on closed interval, along withppredictorsXia,a=1,…,p • Ba(t) summarizes partial effect of XaonYi(t) • Ul(t)are mean-zero random effect functions • Ei(t)are mean-zero residual error functions • Note: tcan be a multi-dimensionalt ∈ℜd

5. Discrete Version of FMM Functional Data on Fine Grid: Suppose each observed curve is sampled on a common equally-spaced grid of length T, withTvery large (hundreds to millions) • Columns index grid points on function t • FMM for higher dimensional functions can also written in this form • E.g. Images: Vectorizeeach T1xT2image: Yi=vec{Yi} • Stack into a single response matrix Y(T=T1×T2 cols), with each row a vectorized image

6. Isomorphic Modeling of Functional Mixed Models (ISO-FMM) • Isomorphic FMM: General multi-domainFMM approach • Key idea: transform data into alternative domain using isomorphic transformation, model FMM in that domain, and then transform results back to original domain for inference • Isomorphic transformation: lossless, invertible transformation that preserves all information in original object; given isomorphic transform h, we haveh-1{h(y)}=y • Examples: Fourier, Wavelets, Splines, Empirically Determined Basis Functions (e.g. FPC) • Why model in alternative domain? • Parsimonious, computational efficient modeling • May be natural way to induce regularization • Estimation and inference obtained for all domains

7. Gaussian Wavelet-Space FMM (Morris and Carroll 2006) Spike-Gaussian mixture prior Adaptive regularization of fixed and random effects Ba(t) and Ul(t) Covariance assumptions accommodate nonstationary local autocorrelation features in functional random effects and residuals Model fit using MCMC after (automatic) vague proper priors, then posterior samples transformed back to original data space for inference.

8. Limitations of G-WFMM • Numerous drawbacks of Gaussian assumptions for random effects uljk and curve-to-curve residuals eijk • May not fit data: generally we might expect heavier tails, especially in wavelet space? • Estimates of random effect functions Ul(t) sensitive to outliers in Ei(t), both globally and locally (int) • Estimates of fixed effect functions Ba(t) sensitive to outliers in Ul(t) and Ei(t), both globally and locally • Exchangable Gaussians for uljk across random effects l leave room for improved adaptive smoothing of Ul(t)

9. Limitations of G-WFMM • Spike-Gaussian mixture for fixed effects Bajk not optimal for variable selection/shrinkage of Ba(t) • New variable selection/sparsity priors have been developed that may have better properties • Spike-DE mixture (Johnstone & Silverman 2004) • Normal/Exp-Gamma (NEG, Griffin & Brown 2005) • Normal/Gamma (NG, Griffin & Brown 2010) • Horseshoe (NIB, Carvalho, Polson & Scott 2010) • Goal: Develop WFMM method robust to outliers and yielding more adaptive estimates and inference for Ba(t) and Ul(t)

10. Robust WFMM: General Model Individual Scale parameters Population Scale parameters • Specify scale mixing distributions g1 for λ, φ, and ψ • Ind. scale parameters serve as wavelet-space outlier weights • Corresponding marginalized distributions are heavier-tailed • Replicates inform estimation of νλ, νφ & νψ for each (j,k), leading to strength borrowing and flexibility/adaption in wavelet space • General Principal: In specifying g1&g2 for λ,φ,&ψ, avoid priors with lighter tails than likelihood.

11. Robust WFMM: EG Implementation • We choose g1 to be Exponential for λ, φ, and ψ • We choose g2 to be Gamma for λ, φ, and ψ • Hyperparameters chosen to be vague and proper • For eijkanduljk behaves like DE across i&l, NEG across (j,k) • ForBajk like Spike-DE across k, Spike-NEG across (a,j,) • Distribution in data space (on grid) is mixture of DE with different scale parameters, which is multivar.& handles nonstationarycovariances

12. Robust WFMM: Model Fitting • Block Gibbs Sampler to fit Bayesian Model • Fixed effectsB*: spike-Gaussian mixture (integrate out u*) • Random effectsu*: Gaussian • Individual scale parameters λ,φ,&ψ: Inverse Gaussian • Population scale parameters νλ, νφ & νψ : Conjugate Gamma • Variable selection parameters γ &π: Bernoulli & Beta • For data tried, MCMC mixed well and results not sensitive to vague hyperpriors

13. Outlier Diagnostics Individual scale parameters can be used to construct global and local outlier diagnostics for individual curves and random effect units Global Outlier Scores: Local Outlier Scores: Standard summary plots and outlier diagnostics can be applied (e.g. Median+/-1.5*IQR)

14. Cancer Proteomics Application • Experiment:16 nude mice with one of two cancer cell lines (A375P/PC3MM2) implanted in one of two organs (lung/brain). • A375P: melanoma cell line/low metastatic potential • PC3MM2: highly metastatic prostate cancer cell line • For each mouse, 2 MALDI-TOF proteomic mass spectra obtained (low/high laser intensity) • Goal:Identify protein/peptide peaks associated with Cell line, organ, and interaction. • Pool information across spectra from both laser intensity settings

15. Application: FMM Specification Let Yi(t) be the MALDI-TOF spectrum i • Xi1=1 if lung/A375P, 0 owXi2=1 for brain/A375P, 0 ow Xi3= 1 if lung/PCMM2Xi4=1 for brain/PCMM2 Xi5= 1 if high laser intensity, -1 if low • Bj(t) = overall mean spectrum: treatment group j=1,2,3,4 B5(t) = laser intensity effect function • Zik=1 if spectrum iis from mouse k (k=1, …, 16) Uk(t) israndom effect function for mouse k. • Take contrasts of fixed effect curves to assess organ and cell line main effects and organ x cell line interaction http://biostatistics.mdanderson.org/Morris

16. Application: Results

17. Application: Outlier Diagnostics

18. Simulation Study • Designed simulation study comparing performance of G-WFMMand R-WFMM • Based on real MALDI-MS data set, so simulated functional data have complex local features just like real data encountered in practice. • Data simulated with random effects and residuals with tails of varying degrees: Normal, DE, t3, t2, t1(Cauchy) • Posterior means for Ba(t)and Ul(t)assessed: • IMSE= integrated variability of PM about truth • IPV= integrated variability of post. samples about PM • ITV= integrated variability of pos. samples about truth • Relative Efficiency REIMSE=IMSEG-WFMM/IMSER-WFMM

19. Relative Efficiency of Robust FMM

20. Cauchy Sim Results: Fixed Effect

21. Cauchy Sim Results: Random Effect

22. Conclusions R-FMM : method for robust functional regression Method developed within the general FMM framework, able to simultaneously model multiple factors and handle correlation between curves through random effect functions. Can be applied to higher dimensional functions like image data. Wavelet domain modeling approach accommodates spiky, irregular functions Other isomorphic transforms could be used, with thought given to assumptions on covariances, distributions, and prior distributions. Automated code scales up to very large data – can just specify Y, X, and Z; rest of method runs automatically.

23. Conclusions General robust approach can be applied with various choices of mixing distributions We used Exp-vague Gamma for residuals and random effects and Spike-Exp-vague Gamma for fixed effects. Simulations: Robust enough even for Cauchy data, yet with little efficiency loss for Gaussian Yielded estimates of fixed and random effects with outstanding adaptive properties, showing remarkable ability to remove spurious features while retaining/recovering true function features

24. Robust Functional Mixed Models • Robust: Can handle heavy tails in residuals, random or fixed effects • Efficient: Gives away very little when data Gaussian, does MUCH better when not. • Adaptive: Evidence of even more adaptive regularization for fixed and random effects induced by flexible mixture of DE • Quick and Automatic: Method automatic, and still fast enough for large data sets (2-4 times slower than G-FMM), still feasible to fit to very large real data sets • Future Directions: Are continuing to extend framework to perform classification, use other isomorphic transforms, handle functional predictors, relax linearity assumptions, deal with super large data sets (e.g. 300GB)