1 / 29

Software Rejuvenation: Analysis, Module and Applications

Software Rejuvenation: Analysis, Module and Applications. Yennun Huang Chandra Kintala Nick Kolettis N. Dudley Fulton. Chris L. Del Checcolo. For the lecture. What is Software Rejuvenation? Why is it needed Probabilistic state transition models Examples. What is Software Rejuvenation?.

miach
Download Presentation

Software Rejuvenation: Analysis, Module and Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Rejuvenation:Analysis, Module and Applications Yennun Huang Chandra Kintala Nick Kolettis N. Dudley Fulton Chris L. Del Checcolo

  2. For the lecture • What is Software Rejuvenation? • Why is it needed • Probabilistic state transition models • Examples

  3. What is Software Rejuvenation? • The act of gracefully terminating an application and immediately restarting • Goal: Prevents unexpected error termination by terminating the program before it suffers an error

  4. Intended Use • Software rejuvenation is primarily indicated for servers where applications are intended to run indefinitely without failure

  5. Why do applications fail? • Process Aging: gradual degradation of application performance, over time, that may lead to premature program termination

  6. Causes • Memory leaks • Unreleased file locks • File descriptor leaking • Etc.

  7. Fine, so just fix the program! • Not always possible, bugs may lie in • the application • the libraries that the application is using • Or in the OS itself • (Except for Windows of course)

  8. Reactive vs. Proactive • Some strategies involve watching to see if an application fails then restarting it. • While not a bad solution, want ability to prevent errors so as to minimize any collateral damage

  9. Software Rejuvenation • Periodic preemptive rollback of continuously running applications to prevent failures in the future

  10. Transition Model For SW without Rejuvenation Transition Model For SW with Rejuvenation

  11. Downtime and cost without rejuvenation • Pf = • Downtimew/o r(L) = Pf * L • Costw/o r(L) = Pf * L * cf

  12. Pp = Pf = Pr = P0 = Downtimew r(L) = (Pf + Pr) * L Costw r(L) = (Pf * cf + Pr * cr) * L Downtime and cost with rejuvenation

  13. Thresholds - Goal Goal is to stay in S0 for the longest amount of time

  14. Thresholds cont. • To see how r4 affects downtime and cost, lets differentiate the previous equations with respect to r4

  15. Thresholds cont. • Downtime: • If r3 is dominant, the derivative becomes negative and downtime decreases when r4 increases thus rejuvenate at state Sp • If r3 is small, slow recovery from SR, downtime increases as r4 increases

  16. Thresholds cont. • Cost: • When cr is dominant, cost increases as r4 increases, implies no rejuvenation benefit • When cr is small, cost decreases as r4 increases

  17. I swear this is the last thresholds slide cont. • Overall, costs need to be calculated for individual programs • For best results: perform rejuvenation at state SP (r4 = ∞) or don’t perform rejuvenation (r4 = 0)

  18. Example 1 • MTBF = 12 months;  = 1/(12*30*24) • Takes 30 min to recover from unexpected error; r1 = 2 • Base Longevity is seven days; r2 =1/(7*24) • If rejuvenation is performed, mean repair time after rejuvenation is 20 minutes; r3 = 3 • Ave. Cost of unscheduled downtime due to failure, cf, is $1,000/hour • Ave. Cost of scheduled downtime during rejuvenation, cr, is $40/hour

  19. Example 2 • MTBF = 3 months;  = 1/(3*30*24) • Takes 30 min to recover from unexpected error; r1 = 2 • Base Longevity is three days; r2 =1/(3*24) • If rejuvenation is performed, mean repair time after rejuvenation is 10 minutes; r3 = 6 • Ave. Cost of unscheduled downtime due to failure, cf, is $5,000/hour • Ave. Cost of scheduled downtime during rejuvenation, cr, is $5/hour

  20. Example 3 • MTBF = 3 months;  = 1/(3*30*24) • Takes 2 min to recover from unexpected error; r1 = 0.5 • Base Longevity is 10 days; r2 =1/(10*24) • If rejuvenation is performed, mean repair time after rejuvenation is 10 minutes; r3 = 6 • Ave. Cost of unscheduled downtime due to failure, cf, is $5,000/hour • Ave. Cost of scheduled downtime during rejuvenation, cr, is $5/hour

  21. Implementation • Implementation of Software Rejuvenation is fairly easy. • Cron Jobs can be set to restart the application at various intervals • watchd can be used to detect if applications have failed and restart them

  22. Real World Examples • BILL-DATS II Collector • Billing collection system used by AT&T long-distance network • Set to rejuvenate after 1 week • Hasn’t prematurely failed after several year

  23. “S” Scientific Speech synthesis system • Long running scientific application • Used to process several hundred sentences over the course of many days • Found to fail after 100 sentences • Rejuvenates after 15

  24. Conclusions: • Decision to use Software Rejuvenation depends on predetermined failure rates and associated costs. • r4 = 0 , No rejuvenation • r4 = ∞ , Rejuvenation

More Related