210 likes | 350 Views
Software Fault Tolerance In A Clustered Architecture : Techniques & Reliability Modeling. Hüsnü Şensoy. Agenda. Introduction RCC Principal Techniques & Architecture Assumptions Reliability Techniques Reliability Modeling & Analysis Conclusion. Introduction.
E N D
Software FaultToleranceIn A ClusteredArchitecture:Techniques & ReliabilityModeling Hüsnü Şensoy
Agenda • Introduction • RCC Principal Techniques & Architecture Assumptions • Reliability Techniques • Reliability Modeling & Analysis • Conclusion
Introduction Software Fault Tolerance In A Clustered Architecture:Techniques & Reliability Modeling
AvailabilityInClusteredEnvironment 4+2 Configuration
RCC Principal Techniques & Architecture Assumptions Software Fault Tolerance In A Clustered Architecture:Techniques & Reliability Modeling
Clustered Architecture Reliability • Error detection • Consequent recovery actions • Data backup Application Application Application Application Database Database Database Database OS OS OS OS Commercial Hardware Commercial Hardware Commercial Hardware Commercial Hardware • Error Detection • Switchover
ZoomInTo A ProcessingNode • RCC Platform Assets • WatchDogInterface • State Server • ClusterManagement • ProcessMonitoring • ResourceMonitors: Disk, Network • RCC AwareApplication • Network Systems’ Applications Off-the-shelfApplications Standard Libraries RCC Libraries Commercial Mirroring/Journaling File System Software Commercial UNIX OperatingSystem Disk Mirror PseudoDriver Commercial UNIX Sytem Hardware Drivers
ReliabilityTechniques Software Fault Tolerance In A Clustered Architecture:Techniques & Reliability Modeling
ReliabilityDimensions Availability Data Consistency
Levels Of Reliability • Level 0: Basic automatic fault detection by watchdog, no automatic fault recovery, no data consistency • A small set of fault classes – hardware & software – is detected by the watchdog. • For a hardware fault, the system is manually reconfigured. • For a software fault, the application process is restarted at the initial internal state which will require initialization of the faulty processor since the application may leave its data in an inconsistent or incorrect state. • Level 1: Basic automatic fault detection by watchdog, automatic fault recovery, no data consistency • A small set of fault classes – hardware & software – is detected by the watchdog & recovery is automatic. • When a fault is detected by the watchdog, the system is automatically recovered – reconfigured for hardware faults and initialized for software faults. • Level 2: Level 1 plus enhanced automatic fault detection by watchdog plus periodic checkpointing, logging & recovery of internal state. • The watchdog & application are enhanced to automatically detect a larger set of fault • The internal state of the application process is periodicallycheckpointed. • After a hardware failure is detected, the system is reconfigured around thefaulty unit. • The application is restarted at the most recent checkpointedinternal state • Level 3: Level 2 plus persistent data recovery. (this is the highest level achievable with RCC) • The persistent data of the application is replicated on a backup disk connected to a backup node, and is kept consistent with the data on the primary node throughout the normal operation of the application. • In case of a fault, in backup node, the backup disk brings the application’s persistent data as close to the state at which the application crashed as possible. • Level 4: Continuous operation without interruption • This level of reliability is not achievable with the RCC.
ReliabilityModeling & Analysis Software Fault Tolerance In A Clustered Architecture:Techniques & Reliability Modeling
Basic Model For Software FaultTolerance Volatile Data Recovery FaultDetection & Recovery Persistent Data Recovery Working Failed
Level 0 Reliability Working Failed %99,96
Level 1 Reliability FaultDetection & Recovery Working Failed %99,98
Level 2 Reliability Volatile Data Recovery FaultDetection & Recovery Working Failed %99,99
Level 3 Reliability Volatile Data Recovery FaultDetection & Recovery Persistent Data Recovery Working Failed ~%100
Conclusion Software Fault Tolerance In A Clustered Architecture:Techniques & Reliability Modeling
Conclusion In this work, • a RCC has been proposed. • Different levels of reliability have been defined. • A reliability analysis is held via Markov modelling.
? Questions & Comments Software Fault Tolerance In A Clustered Architecture:Techniques & Reliability Modeling