Approximations of the population Fisher information matrix- differences and consequences Joakim Nyberg, Sebastian Uecker

1. Approximations of the population Fisher information matrix- differences and consequencesJoakim Nyberg, Sebastian Ueckert, Andrew C. Hooker

2. Background • At PODE 2009 all Population Optimal Design (OD) • Software should evaluate the same simple Warfarin problem… • 1-compartment model, 1st order absorption, oral dose 70 mg • Proportional error model (σ2=0.01) • 32 subjects with 8 measurements at 0.5, 1, 2, 6 ,24, 36, 72,120 hours (evaluation)

3. Fisher Information Matrix (FIM) FIM can be calculated in different ways: Assuming var(y) w.r.t. the fixed effects=0 Assuming var(y) w.r.t. the fixed effects≠0 A* is somewhat modified/updated if full is used, i.e. Different between full and reduced

4. Fisher Information Matrix (FIM) The FIM • If we have correlation between fixed effects • and random effects in like the FULL is the “theoretically • correct” method. • If not, the Reduced is “correct theoretically” but this is seldom the case in Pharmacometrics

5. Results from last PODE 2009 The “truth” * * * Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003

6. Results from last PODE 2009 The “truth”

7. Results from last PODE 2009 Possibly issues with the Cramér-Rao inequality

8. Results from last PODE 2009 summary • Software gave similar results with similar approximations • Reduced superior to Full in terms of predicting the “truth” • Even less predictive performance with higher order FOCE-based FIM.

9. Possible reasons – Initial ideas • The derivation of Full or Red is wrong • Derive FIM with simulations, i.e. integrate over observed FIM • FO-approximation too poor - FOCE is obviously not enough, try high order approximations • Asymptotic behavior (FIM-1≤COV) - Increase data set x 2 => SE should decrease by 2(1/2) • Numerical instability in Full but not in Red FIM - Using automatic differentiation (AD) to avoid step length issues • Estimation software is not true ML-estimator, • i.e. efficiency of estimator not accurate • - NM hard to know how the parameter search is performed • but Monolix well documented

10. Investigations – Reducing the complexity • ln-transform model to have additive res-error (avoiding interaction terms) • Check that the problem holds for prop IIV structure (FO approximation => proportional IIV = exp IIV) • Fix all parameters except fixed effect Ka

11. Results – Reducing the complexity ln model, add error, exp IIV = prop IIV • Issues still remaining => work with simplified model * 100 000 bootstrap samples ** Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003

12. Results – Full vs Reduced • Asymptotic behavior (FIM-1≤COV) • Increase data set x 2 => SE should decrease by 2(1/2) • Numerical instability in Full but not in Red FIM - Using automatic differentiation (AD) to avoid step length issues None of this affected the results (2 down 3 to go)

13. Next things to try... • The derivation of Full or Red is wrong • Derive FIM with simulations, i.e. integrate over observed FIM • FO-approximation too poor - FOCE is obviously not enough, try high order approximations SO

14. Similar SO – Closer to truth Simulation based FIM FOCE from NONMEM solve the problem! Results – High order approximations & simulation based derivations * 100 000 bootstrap samples

15. Results Full vs Red • The derivation of Full or Red is wrong • Integration FIM ≈ Analytic FIM => Not the answer • FO-approximation too poor • SO shrinks the differences but still to poor of an approx. • FOCE is worse but NONMEM integrated FOCE FIM is good?! • Possibly issues with the FOCE method?

16. FOCE FIM – Differences & Improvements • NONMEM FOCE assumes linearization around the mode • of the distribution => correlation between the individual • parameters and the population parameters. • Analytic FIMFOCE * does not assume this • To calculate individual mode data is needed • Update Analytic FIMFOCE to include the correlation: • Calculate Expected Empirical Bayes Estimates (EEBE) EEBE are not data dependent • Whenever PopED differentiates pop parameters; differentiate EEBE as well * Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003

17. Results – Updated FIMFOCE • The new FOCE method solves the problem!

18. The answer • The Full FIM does not always work with the FO-approximation

19. When to use which method?

20. Does Full/Red affect the optimal design? Full Reduced 6 2 2 2 1 1 1 1 3 support points 5 support points

21. Does linearization method affect the optimal design? Surface of |FIM| for SOCE-MC SOCE>MC SOCE=MC SOCE<MC Optimal MC Optimal SOCE Example from PAGE 2009 Nyberg et al

22. Always use reduce FIM instead? Full |FIM| Reduced |FIM| Similar results with the transit compartment model Results from Nyberg et al PODE 2008

23. Surface of RSE(%) – Full FIM only different from reduced in some regions Full Ka RSE(%) Reduced Ka RSE(%) S3-S7: (7.79h) S3-S7: (7.79h) S8: (120h) S8: (120h) Full ≈ Red

24. Conclusions • The FO-approximation is not always enough for Full FIM Possibly also too poor approx. for Reduced • Reduced FIM collapses occasionally • High order approximations stabilize differences • Different approximations give different optimal designs, e.g. different sampling times and different number of support points

25. Suggestions • If runtime allows – Use high order approximations FOCE, SO, SOCE, MC etc. • If Red is stable – Use reduced to optimize but evaluate with both • If Red is unstable – Optimize with Full but evaluate with Red Beware: - No “golden” solution is presented - The Cramer-Rao inequality does not hold comparing different methods when optimizing / estimating - To get “correct” SE from the FIM either sim/est needs to be performed or high order FIMs need to be evaluated

26. Thank you I would like to acknowledge Sergei Leonov for our interesting emails discussing these issues.